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PREPARING  BRIGADE  COMBAT  TEAM  SOLDIERS  FOR  MISSION  READINESS  BY 
WAY  OF  INTANGIBLE  PSYCHOLOGICAL  CONSTRUCTS  AND  THEIR  APPLICATIONS 
PHASE  II:  MEASURMENT  AND  LEARNING  METHODS 

EXECUTIVE  SUMMARY 


Research  Requirement: 

This  report  follows  an  initial  report  (Phase  I)  that  focused  on  the  identification  of 
psychological  constructs  critical  for  operational  units  to  train  prior  to  deployment.  This  second 
report  focuses  on  instrument  development  as  well  as  the  identification  of  effective  learning 
methods  relevant  to  training  intangibles.  A  further  objective  was  the  identification  of  germane 
training  events  that  would  lend  themselves  to  training  and  measuring  the  intangibles  critical  to 
operational  units. 

Procedure: 

As  in  the  first  phase,  both  academic  and  military  sources  were  reviewed  to  support 
instrument  development  and  identification  of  applicable  learning  methods.  Soldiers  and  leaders 
were  interviewed  and  completed  materials  to  assist  in  the  initial  stages  of  instrument 
development.  Soldiers  were  also  asked  to  identify  training  events  most  relevant  for  training  and 
measuring  intangibles  as  well  as  how  current  training  events  lend  themselves  to  developing 
intangibles. 

Findings: 

The  literature  focused  on  existing  measures  of  the  primary  intangibles  that  were 
identified  in  Phase  I  of  this  research.  Those  intangibles  included:  Initiative,  Will,  Grit,  and 
Hardiness.  Examination  of  the  literature  related  to  measures  of  psychological  constructs 
revealed  primary  intangibles  consisted  of  sub-components.  For  instance,  Hardiness  has  been 
argued  to  consist  of  three  components:  commitment,  control,  and  challenge.  The  specificity  of 
this  review  of  available  measures  informs  the  follow-on  research  to  develop  a  construct  valid 
measure  encompassing  the  primary  intangibles.  Soldiers  and  leaders  identified  such  training 
events  as  combat  training  center  exercises  and  well-planned  field  exercises  where  intangibles  are 
often  exhibited  and  provide  an  opportunity  to  further  train  the  critical  intangibles. 

Utilization  and  Dissemination  of  Findings: 

The  findings  from  this  second  phase  of  research  will  assist  in  the  development  of  a  valid, 
field-expedient  measure  that  can  be  used  by  a  variety  of  operational  units  during  existing  and 
future  training  events.  The  identification  and  measure  of  psychological  constructs,  as  they  relate 
to  training  of  existing  tangible  skills,  will  enhance  individual  Soldier  development  and,  thus, 
overall  unit  readiness  as  operational  units  prepare  for  deployment. 
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PREPARING  BRIGADE  COMBAT  TEAM  SOLDIERS  FOR  MISSION  READINESS  BY 
WAY  OF  INTANGIBLE  PSYCHOLOGICAL  CONSTRUCTS  AND  THEIR 
APPLICATIONS  PHASE  II:  MEASURMENT  AND  LEARNING  METHODS 


The  goal  of  the  Brigade  Combat  Team  (BCT)  Preparatory  Skill  Set  research  is  to  develop 
content  valid  measures  of  intangible  psychological  constructs  and  identify  effective  methods  for 
measuring  intangibles  during  training.  The  U.S.  Anny  Research  Institute  for  the  Behavioral  and 
Social  Sciences  (ARI)  has  used  the  term  intangible  to  describe  psychological  constructs  that 
contribute  to  Soldier  mission  readiness.  Soldier  mission  readiness  describes  Soldiers’ 
preparedness  for  all  missions  (e.g.,  U.S.  based  disaster  relief,  short-term  contingency  operations, 
long  tenn  deployments,  counterinsurgency  operations).  During  Phase  I  of  the  research,  a 
literature  review  and  data  collection  were  conducted  to  identify  the  most  critical  intangibles  to 
Soldier  mission  readiness.  The  data  collection  also  obtained  BCT  Soldier  and  leader  assessments 
of  the  current  state  of  intangibles  training  and  methods  by  which  intangibles  are  currently 
trained. 


Phase  II  of  this  research  focuses  on  the  identification  of  effective  learning  methods  and 
measurement  tools  of  intangibles  for  mission  readiness  training  and  methods  to  adapt  research- 
derived  measures  for  field  use.  The  purpose  of  the  Phase  II  literature  review  is  to  guide  the 
design  of  measures  and  an  overall  training  strategy.  The  Phase  II  literature  review  builds  on 
Phase  I  by  reviewing  journal  articles  and  technical  reports  concerning  the  measurement  of  the 
most  critical  intangibles  identified  in  Phase  I  (i.e.,  hardiness,  grit,  will,  and  initiative). 
Additionally,  the  Phase  II  literature  review  examines  training  methods  and  measurement 
implementation  using  Phase  I  data  collection  findings.  Lastly,  Phase  II  reviews  the  literature 
concerning  outcome-based  training  and  education,  as  well  as  learning  theories,  to  guide  the 
design  and  implementation  of  measures. 

Literature  Review  of  Existing  Measures  for  Selected  Intangibles 

There  were  four  intangibles  that  were  selected  in  Phase  I  for  further  review  in  Phase  II: 
initiative,  will,  grit,  and  hardiness.  This  section  identifies  the  most  effective  existing  measures 
used  for  training  these  intangibles.  Existing  measures  will  be  described  in  detail  in  this  section 
and  recommendations  will  be  made  about  the  suitability  and  modification  of  the  measures  for  use 
in  BCT  readiness  training.  If  an  intangible  does  not  have  a  reviewable  existing  measure, 
doctrine  and  other  relevant  literature  will  be  reviewed  to  identify  important  concepts  and  content 
that  could  be  used  in  the  design  of  a  measure  for  that  intangible. 

The  following  sections  examine  the  intangibles  individually  and  discuss  conceptual 
overlap  between  the  intangibles.  The  discussion  identifies  similarities  and  differences  among  the 
intangibles.  The  goal  of  this  discussion  is  to  address  the  appropriateness  of  combining  some 
intangibles  into  one  measure  or  provide  rationale  for  measuring  each  intangible  individually. 
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Initiative 


The  Army  defines  initiative  as  “the  willingness  to  act  in  the  absence  of  orders,  when 
existing  orders  no  longer  fit  the  situation,  or  when  unforeseen  opportunities  or  threats  arise” 
(Department  of  the  Army,  2006a,  p.  4-5).  Frese,  Kring,  Soose,  and  Zempel  (1996)  stated  that 
initiative:  “1)  is  consistent  with  the  organization’s  mission,  2)  has  a  long  tenn  focus,  3)  is  goal 
directed  and  action  oriented,  4)  is  persistent  in  the  face  of  barriers  and  setbacks,  and  5)  is  self 
starting  and  proactive”  (p.  38).  While  the  Anny  definition  describes  the  situations  of  when  to 
apply  initiative,  the  definition  from  Frese  et  al.  describes  the  different  components  of  initiative  in 
a  general  context.  Further,  Frese  et  al.  have  teamed  with  other  researchers  to  develop  multiple 
reliable  and  criterion  valid  measures  of  initiative. 

Frese,  Fay,  Hilburger,  Leng,  and  Tag  (1997)  demonstrated  the  convergent  validity  of 
their  seven  item  self-report  initiative  scale  with  other  operationalizations  of  initiative  (see  seven 
items  below). 

•  I  actively  attack  problems. 

•  Whenever  something  goes  wrong,  I  search  for  a  solution  immediately. 

•  Whenever  there  is  a  chance  to  get  actively  involved,  I  take  it. 

•  I  take  initiative  immediately  even  when  others  don't. 

•  I  use  opportunities  quickly  in  order  to  attain  my  goals. 

•  Usually  I  do  more  than  I  am  asked  to  do. 

•  I  am  particularly  good  at  realizing  ideas. 

In  addition  to  finding  convergent  validity  with  other  measures  of  initiative,  their  measure  of 
initiative  is  also  related  to  components  of  initiative  such  as  action  orientation  and  overcoming 
barriers.  Both  of  which  are  a  part  of  the  Anny  definition  of  initiative. 

Bledow  and  Frese  (2009)  further  examined  this  seven  item  self-report  initiative  scale  in  a 
workplace  setting  to  create  a  better  measure  of  initiative  and  address  previous  concerns  over  the 
construct  validity  of  a  self-report  measure  of  initiative.  Specifically,  Frese  et  al.  (1997)  first 
identified  that  a  self-report  measure  of  initiative  captures  the  importance  people  assign  to 
initiative,  rather  than  measuring  how  much  initiative  a  person  actually  possesses  through 
observing  initiative  behaviors  in  others.  Frese  et  al.  also  identified  a  similar  problem  with 
Likert-type  measures  of  initiative.  Initiative  is  defined  on  the  level  of  observable,  situated 
action.  A  Likert  measure  of  initiative  is  problematic  because  it  captures  the  importance  one 
attaches  to  initiative,  but  not  actual  initiative  as  displayed  through  one’s  behaviors.  This  research 
suggests  the  following  design  features:  1)  do  not  rely  on  self  report  and  2)  measure  initiative 
through  observed  behaviors.  These  features  will  improve  the  construct  validity  of  initiative 
measures. 

As  part  of  Bledow  and  Frese’s  (2009)  effort  to  improve  on  the  measurement  of  initiative, 
they  created  three  measures:  a  seven  item  self-report  initiative  scale,  a  seven  item  supervisor 
initiative  scale,  and  a  situational  judgment  test  (SJT)  for  initiative.  The  purpose  of  Bledow  and 
Frese’s  study  was  to  design  the  SJT  and  demonstrate  both  its  reliability  and  validity.  The  SJT 
displayed  an  acceptable  test-retest  reliability  (i.e.,  Pearson  correlation)  of  r=.73.  The  SJT 
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demonstrated  criterion-related  validity  as  the  measure  was  significantly  related  to  perfonnance 
r=.48,  p<. 0 1 .  The  measure  of  perfonnance  used  was  a  three  item  measure  of  overall  employee 
performance  developed  by  Motowidlo  and  Van  Scotter  (1994). 

Bledow  and  Frese  (2009)  also  designed  a  measure  of  initiative  for  use  by  supervisors  of 
employees.  The  supervisor  measure  is  essentially  the  same  as  the  self  report  measure  of 
initiative  except  the  subject  of  the  item  changes  (e.g..  This  employee  actively  attacks  problems). 
The  supervisor  measure  was  the  most  reliable  measure  a  =  .91  and  the  measure  with  the  greatest 
evidence  of  criterion-related  validity  when  conelated  with  measures  of  performance,  r=.72, 

p<.  01. 


The  brevity  of  the  supervisor  measure  (i.e.,  seven  items)  would  make  it  easy  to 
implement  and  require  fewer  resources  than  a  SJT.  The  supervisor  measure  of  initiative  has  also 
been  shown  to  possess  sufficient  reliability  and  criterion  validity  (Bledow  &  Frese,  2009). 
Further,  because  the  researchers’  operationalization  of  initiative  is  consistent  with  the  Anny’s 
definition  of  the  construct,  the  seven  item  supervisor  measure  would  require  only  minimal 
changes  for  use  as  an  Army  training  assessment  instrument.  By  applying  doctrine,  literature 
reviewed  in  Phase  I  and  Anny  subject  matter  experts  the  measure  could  be  revised  to  achieve 
face  validity  for  BCT  commanders,  training  support  personnel,  and  Soldiers. 

Will 

The  Anny  defines  will  as  the  “inner  drive  that  compels  [Soldiers]  to  keep  going  even 
when  exhausted,  hungry,  afraid,  cold,  and  wet”  (Department  of  the  Anny,  2006b,  p.  5-3).  Anny 
doctrine  effectively  defines  will  and  describes  its  features.  However,  there  were  no  journal 
articles  that  operationalized  will  in  a  measure.  Therefore,  the  following  discussion,  including 
implications  for  the  present  research,  is  drawn  from  doctrine. 

Anny  doctrine  states  that  a  Soldier’s  “will  in  conjunction  with  self-discipline  and 
confidence  helps  him  or  her  do  what  is  right;  even  when  it  might  be  difficult”  (Department  of  the 
Anny,  2006b).  That  quote  highlights  the  importance  of  self-discipline  and  confidence  and 
suggests  a  moderating  relationship  between  will,  confidence,  and  self-discipline.  Therefore,  a 
measure  of  will  should  include  items  concerning  confidence  and  self-discipline  to  account  for 
and  understand  the  relationship  between  the  three  intangibles. 

Doctrine  endorses  the  idea  that,  “commitment  to  beliefs  such  as  Warrior  Ethos,  Anny 
values,  justice,  liberty,  freedom,  and  motivation  are  helpful  in  developing  a  Soldier’s  will” 
(Department  of  the  Army,  2006b).  As  Warrior  Ethos  is  described  in  the  literature  it  seems 
related  to  will  and  also  provides  insights  into  the  measurement  of  will.  The  Anny  defines 
Warrior  Ethos  as  a  “shared  sentiment  internal  to  Soldiers  that  represents  the  spirit  of  the 
profession  of  arms”  (Department  of  the  Army,  2006b).  The  following  four  tenets  convey  this 
idea  (Department  of  the  Army,  2008): 

•  I  will  always  place  the  mission  first. 

•  I  will  never  accept  defeat. 

•  I  will  never  quit. 

•  I  will  never  leave  a  fallen  comrade. 
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These  tenets  are  conceptually  related  to  the  Anny’s  definition  of  will.  Further,  Riccio, 
Sullivan,  Klein,  Salter,  and  Kinnison  (2004)  and  Klein,  Salter,  Riccio,  and  Sullivan  (2006) 
examined  the  components  of  Warrior  Ethos  and  identified  two  components  that  seem  to  be 
related  to  will.  The  first  component  is  perseverance  -  “ability  to  work  through  adversity,  to 
persevere  at  all  times,  and  to  embody  each  of  the  four  tenets  of  Warrior  Ethos”  (Klein  et  ah, 

2006,  p.  3).  The  second  component  is  ability  to  adapt  -  “flexibility  and  smooth  reaction  to 
changes  in  mission  and  unexpected,  often  unpleasant,  surprises  whether  from  enemy  contact, 
change  in  weather  or  terrain,  or  change  in  mission  from  combat  to  stability  and  support 
operations  and  back”  (Klein  et  ah,  2006,  p.  3).  While  Warrior  Ethos  is  comprised  of  a  number  of 
components,  each  of  them  appears  related  to  will.  The  tenets  of  Warrior  Ethos,  the  definition  of 
Warrior  Ethos,  and  the  description  of  its  components  provide  insight  into  concepts  related  to  will 
and  should  be  considered  in  the  design  of  a  measure  of  will.  What  remains  is  to  translate  the 
tenets  of  Warrior  Ethos,  together  with  those  of  confidence  and  self-discipline,  into  descriptive 
statements  of  observable  phenomena  in  the  field. 

Grit 


Grit  is  defined  as  a  “perseverance  and  passion  for  long-term  goals”  (Duckworth, 

Peterson,  Matthews,  &  Kelly,  2007,  p.  1087).  The  distinction  between  grit  and  other  intangibles 
is  that  grit  focuses  primarily  on  the  achievement  of  long-tenn  goals.  When  challenges  or 
adversity  are  discussed  in  the  literature,  the  associated  length  of  time  for  an  individual  to  display 
grit  can  occur  for  months  or  years.  Duckworth  stated  that  passion  is  a  factor  related  to  grit  that 
helps  a  person  sustain  interest  in  their  long  tenn  goal  over  time. 

In  2007,  Duckworth,  et  al.  developed  a  measure  called  the  grit  scale.  The  grit  scale 
contained  two  subscales:  consistency  of  interest  and  perseverance  of  effort.  As  part  of  their 
validation  process,  they  tested  the  measure  with  several  different  samples  (e.g.,  open  survey 
participation  online,  undergraduate  students,  and  freshman  cadets  at  West  Point).  In  the  West 
Point  study,  results  indicated  that  the  grit  scale  was  reliable  a=.79.  Using  binary  logistic 
regression  Duckworth,  et  al.  demonstrated  the  grit  scale’s  validity.  The  grit  scale  was  most 
effective  at  predicting  the  completion  of  a  rigorous  summer  training  course.  “Cadets  who  were  a 
standard  deviation  higher  than  average  in  grit  were  more  than  60%  more  likely  to  complete 
summer  training,  /1=.48,  Odds  Ratio  (0f?)=.62,/K.OOl”  (Duckworth  et  al.,  p.  1095). 

In  2009,  Duckworth,  &  Quinn  created  and  validated  a  shorter  version  of  the  original  grit 
scale  using  another  sample  of  West  Point  freshman  cadets.  The  scale  reliability  (a=.77)  was 
acceptable  and  similar  to  the  original  scale  and  the  short  scale  was  more  effective  at  predicting 
completion  of  the  rigorous  summer  training  course.  “Cadets  who  scored  a  standard  deviation 
higher  than  average  on  the  Grit-S  were  99%  more  likely  to  complete  summer  training,  /?=.69, 
OR=1.99,/?<.001”  (Duckworth  &  Quinn,  2009,  p.  171). 

The  ARI  used  the  Duckworth,  et  al.  (2007)  original  self-report  grit  scale  as  part  of  a  study 
that  explored  the  extent  to  which  perseverance  contributed  to  a  Soldier  completing  the  Special 
Forces  Assessment  and  Selection  (SFAS)  process  and  being  selected  for  Special  Forces  (SF) 
training  (Beal,  2010).  The  author  recommended  that  the  grit  scale  not  be  used  on  its  own,  but  in 
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conjunction  with  other  measures  to  inform  and  support  recruiting  and  selection  decisions.  Beal 
drew  several  conclusions  concerning  the  grit  scale  that  are  relevant  to  the  present  research: 

•  The  grit  scale  is  easy  to  administer  and  score. 

•  The  grit  scale  provides  an  empirically-valid  measure  of  perseverance  that  is 
independent  from  all  other  measures. 

•  The  grit  scale  can  be  used  for  professional  development  as  a  marker  of  individual 
levels  of  perseverance,  (p.  14) 

The  complete  list  of  items  on  Duckworth  and  Quinn’s  (2009)  grit  scale  is  presented 
below.  There  are  two  subscales:  consistency  of  interest  and  perseverance  of  effort.  The  items 
that  are  included  in  the  shortened  version  of  the  scale  are  in  italics. 

Consistency  of  Interest 

•  I  often  set  a  goal,  but  later  choose  to  pursue  a  different  one. 

•  I  have  been  obsessed  with  a  certain  idea  or  project  for  a  short  time,  but  later  lost  interest. 

•  I  have  difficulty  maintaining  my  focus  on  projects  that  take  more  than  a  few  months  to 
complete. 

•  New  ideas  and  projects  sometimes  distract  me  from  previous  ones. 

•  My  interests  change  from  year  to  year. 

•  I  become  interested  in  new  pursuits  every  few  months. 

Perseverance  of  Effort 

•  I finish  whatever  I  begin. 

•  Setbacks  don  7  discourage  me. 

•  I  am  diligent. 

•  I  am  a  hard  worker. 

•  I  have  achieved  a  goal  that  took  years  of  work. 

•  I  have  overcome  setbacks  to  conquer  an  important  challenge. 

The  definition  of  grit  as  well  as  the  subscales  and  items  from  Duckworth  and  Quinn’s 
(2009)  grit  scale  should  be  used  as  a  basis  for  a  measure  created  to  support  Anny  BCT’s 
assessment  of  grit.  The  grit  scale  has  been  validated  with  experimental  research  using  multiple 
samples.  And  it  includes  items  that  measure  aspects  of  a  long  term  persistence  that  is  unique  to 
grit.  Only  slight  changes  to  words  and  method  of  assessment,  then,  may  be  needed  to  adapt  this 
measure  of  grit  for  application  by  an  Army  BCT. 

Hardiness 

Hardiness  was  originally  defined  as  “a  personality  attribute  that  reflects  the  courage  and 
motivation  to  cope  effectively  with  the  stressors  of  daily  life”  (Vogt,  Rizvi,  Shipherd,  &  Resick, 
2008,  p.  61).  Maddim  Matthews,  Kelly,  Resurreccion,  and  Villarreal  (2010)  defined  hardiness  as 
“a  specific  set  of  attitudes  and  skills  that  provide  the  courage,  motivation,  and  strategies  leading 
to  resilience  and  growth  in  stressful  circumstances”  (p.  2).  These  two  definitions  are  similar  in 
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that  both  identify  hardiness  as  something  expressed  through  courage  and  motivation,  especially 
in  the  presence  of  stressors. 

“In  a  review  of  hardiness  theory  and  research.  Funk  (1992)  concludes  that  both 
conceptually  and  psychometrically,  the  Dispositional  Resiliency  Scale  (DRS)  provides  the  most 
sound  hardiness  measure”  (Bartone,  Roland,  Picano,  &  Williams,  2008,  p.  79).  The  DRS  is  an 
established  scale  that  has  been  studied  and  used  to  measure  hardiness  in  multiple  military  studies 
(Bartone,  1999;  Bartone  et  ah,  2008;  Bartone,  Ursano,  Wright,  &  Ingraham,  1989).  At  present, 
the  DRS  measure  is  copyrighted,  yet  government  users  are  afforded  free  access  to  the  measure 
provided  they  adhere  to  a  user  agreement  which  stipulates  that  the  measure  may  not  be  altered. 
Though  the  inability  to  alter  the  DRS  makes  it  less  applicable  to  the  design  of  an  Army-relevant, 
behavioral  measure,  the  components  of  the  DRS  scale  provide  a  previously  researched 
conceptual  foundation  that  measures  could  be  built  upon. 

Three  hardiness  components  that  have  been  operationalized  as  reliable  subscales  of  DRS 
are:  commitment,  control,  and  challenge  (Bartone,  1999;  Hull,  Van  Treuren,  &  Vimelli,  1987; 
Kobasa,  1979;  Kobasa,  Naddum  &  Kahn,  1982;  Maddi  et  ah,  2002).  “Commitment  is  the  extent 
to  which  a  person  is  engaged  in  a  variety  of  life  domains,  such  as  family,  friends,  and  work.  The 
control  dimension  of  hardiness  represents  the  extent  to  which  a  person  believes  that  he  or  she  is 
able  to  control  events  that  happen  in  his  or  her  life.  Challenge  reflects  the  extent  to  which  people 
generally  perceive  difficult  situations  as  challenges  rather  than  as  threats”  (Eschleman  & 
Bowling,  2010,  p.278-279). 

While  the  DRS  scale  has  supportive  research  validating  its  effectiveness  in  measuring 
hardiness,  there  are  two  issues  against  using  it  as  a  measure  of  hardiness  in  BCT  pre-deployment 
skills  training  and  assessment.  First,  the  DRS  scale  cannot  be  altered.  The  scale  has  been  shown 
to  be  reliable  and  valid  in  Anny  settings,  but  the  inability  to  tailor  the  scale  creates  a  limitation 
should  an  alteration  be  deemed  necessary.  Second,  the  DRS  scale  measures  hardiness  as  a 
personality  construct.  If  hardiness  is  treated  as  a  personality  construct,  without  associated 
behaviors,  it  will  be  more  difficult  to  observe  and  may  be  perceived  as  not  subject  to  application 
and  development.  For  example,  the  DRS  items  are  measuring  level  of  hardiness  in  the  abstract 
as  opposed  to  measuring  hardiness  with  respect  to  a  specific  task.  In  contrast  to  DRS 
operationalization,  hardiness  could  be  measured  by  way  of  behaviorally  based  items, 
situationally  and  task-based,  where  observation  and  developmental  feedback  are  far  more 
concrete  and  objective.  A  behavioral  instrument  would  also  be  more  easily  applied  to  training, 
achieving  the  project  goal  of  creating  a  measure  for  use  by  an  Army  BCT. 

In  summary,  the  two  definitions  of  hardiness  express  the  construct  in  terms  of  courage 
and  motivation,  especially  in  the  presence  of  stressors.  Further,  the  literature  shows  that 
commitment,  control,  and  challenge  are  important  components  of  hardiness.  These  concepts 
could  serve  as  a  foundation  for  developing  a  behaviorally-based  assessment  instrument 
conducive  to  an  Army  field  training  environment. 
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The  Relationship  between  Intangibles 

The  literature  on  each  of  the  four  intangibles  under  investigation  suggests  both 
similarities  and  differences  among  them.  The  intangibles  are  all  similar  in  that  they  emphasize 
the  importance  of  being  action  oriented  and  taking  action  when  necessary.  Possessing  an  action 
orientation  during  missions  would  help  when  confronting  adversity  and  complexity  (which  are 
characteristic  of  the  current  operational  environment).  Thus,  action  orientation  is  an  important 
and  common  part  of  initiative,  grit,  will,  and  hardiness. 

The  construct  of  initiative  is  the  most  conceptually  different  from  the  other  three 
intangibles.  The  literature  suggests  the  necessity  of  measuring  the  following  components  as  a 
means  for  measuring  initiative:  action  orientation  and  whether  individuals  are  proactive  or  self¬ 
starters.  An  action  orientation  (as  previously  discussed)  is  a  point  of  similarity  between  the  four 
constructs.  Yet  Army  doctrine  also  calls  out  the  importance  of  acting  in  the  absence  of  orders. 
Therefore,  initiative  is  distinct  from  the  other  intangibles  in  that  it  also  focuses  on  one ’s  own 
ability  to  self-start,  proactively  address  situations  before  they  occur,  and  use  the  higher  leader’s 
intent  to  appropriately  confront  challenges  in  the  absence  of  orders. 

Will,  grit,  and  hardiness  are  similar  in  that  perseverance  is  a  key  concept  to  each.  The 
relationship  between  will  and  perseverance  is  evident.  Grit  is  long-term  perseverance  toward  the 
accomplishment  of  long-term  goals.  Similar  to  will,  hardiness  is  perseverance  specifically  in 
moments  of  stress  or  when  resilience  is  required.  Hardiness  is  somewhat  distinct  from  the  other 
intangibles  in  that  it  is  typically  referred  to  as  a  personality  factor  or  a  set  of  attitudes  concerning 
one’s  willingness  to  act.  Will  and  grit  are  more  focused  on  the  quality  or  level  of  perseverance 
in  one’s  actions.  In  this  sense,  hardiness  may  be  a  precursor  to  will  and  grit  or  an  effect  of 
having  been  exposed  to  situations  which  require  will  and  grit.  For  example,  hardiness  is 
something  that  one  could  hear  or  observe  in  a  person’s  attitudes  or  as  a  part  of  their  personality; 
independent  of  stimuli.  Whereas,  grit  and  will  are  more  likely  to  be  observed  as  one  confronts 
adversity.  Therefore,  a  hardiness  measure  might  best  predict  perseverance  in  advance  of  its 
display,  while  measures  of  grit  and  will  better  assess  perseverance  in-the-moment  of  its 
demonstration. 

Review  of  Phase  I  Findings  Related  to  the  Training  Environment 
and  Measurement  Development 

The  Phase  I  literature  review,  together  with  the  quantitative  and  qualitative  data 
collection,  identified  four  intangibles  that  are  most  critical  to  Soldier  mission  readiness  (i.e., 
initiative,  will,  grit,  and  hardiness).  The  following  criteria  were  used  to  select  these  intangibles: 
1)  criticality  to  Soldier  mission  readiness,  2)  expressed  participant  need  for  new  training,  3)  the 
feasibility  of  measurement  and  training,  and  4)  the  importance  placed  on  these  intangibles  in  the 
literature  and  doctrine.  The  findings  that  are  most  relevant  to  the  present  discussion  of  intangible 
training  and  measurement  are  listed  below.  This  data  was  analyzed  for  high  frequency  themes 
according  to  how  often  it  was  mentioned  by  participating  BCT  Soldiers  and  their  leaders. 

One  of  the  most  frequent  themes  from  Phase  I  focus  group  findings  was  that  there  is  not 
enough  time  for  training.  Soldiers  pointed  to  increased  operations  tempo  (OPTEMPO)  and  the 
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shortened  Army  Forces  Generation  (ARFORGEN)  cycle  as  obstacles  to  training.  In  addition  to 
these  concerns,  Soldiers  said  that  they  are  always  receiving  more  to  do  from  headquarters  which 
also  requires  time.  These  findings  establish  significant  time  constraints  that  recommend  against 
the  development  of  any  measure  that  would  be  seen  as  an  additional  requirement.  Therefore,  the 
measurements  should  be  designed  to  be  applied  in  existing  training  exercises  with  simple 
implementation,  minimal  additions  to  training  requirements,  and  obvious  value-added  for  the 
time  required  to  employ  them. 

The  most  frequently  mentioned  desired  training  features  identified  during  the  Phase  I  data 
collection  were  difficult/rigorous/challenging  training,  experiential  training,  realistic  training, 
train  using  repetition,  incorporate  uncertainty  into  training,  and  that  field  exercises  should  train 
Soldiers  to  deal  with  stress.  Additionally,  Soldiers  identified  what  training  environments  are 
most  effective  for  training  intangibles.  The  training  features  that  were  most  frequently 
mentioned  were  those  associated  with  effective  training  environments  in  general:  challenging, 
stressful,  and  realistic  (e.g.,  National  Training  Center  (NTC)  or  large  field  exercises).  Soldiers 
noted  that  training  needs  to  be  challenging,  but  not  impossible.  Training  needs  to  push  Soldiers 
just  beyond  their  limit  so  they  realize  that  they  can  create  new  limits  and  go  beyond  what  they 
thought  they  were  capable  of.  Yet  at  the  same  time,  training  should  not  be  so  difficult  as  to 
invite  continual  failure. 

Training  environments  that  did  not  use  the  desired  training  features  (e.g.,  classroom 
training)  were  seen  as  less  effective  or  ineffective  at  training  intangibles.  However,  there  were 
instances  where  Soldiers  indicated  that  classroom  training  was  a  prerequisite  for  more  advanced 
training.  Additionally,  these  Soldiers  noted  that  a  crawl,  walk,  run  method  was  necessary  for 
training.  Similarly,  doctrine  says  “that  leaders  give  subordinates  complex  tasks  to  gradually 
develop  the  will  necessary  to  take  on  more  difficult  tasks”  (Department  of  the  Army,  2006b). 
Therefore,  intangibles  are  displayed  and  best  evaluated  in  the  most  challenging  training 
environments.  Yet  the  effective  development  of  intangibles  most  likely  begins  well  before 
Soldier  exposure  to  the  most  demanding  training  environments.  Initial  practice  of  tasks  under 
nonnal  operating  conditions  helps  Soldiers  gain  necessary  skills  and  confidence  before  more 
challenging  training  conditions  are  imposed  on  them. 

The  rigors,  challenges,  and  adversities  that  exist  in  difficult  training  events  bring  out  the 
intangibles.  The  initial  research  identified  primary  intangibles  relevant  for  the  U.S.  Anny 
Soldier.  The  research  described  in  this  report  examines  training  events  and  other  methods  where 
intangibles  are  likely  expressed.  Future  research  efforts  could  then  explore  if  and  how  a 
Soldier’s  exhibition  of  intangibles,  a  demonstration  of  their  technical  and  tactical  skills,  and  the 
training  environment  are  linked.  For  these  reasons  a  measure  of  an  intangible  needs  to  also 
measure  Soldier  training  perfonnance  and  training  outcomes.  Linking  intangibles  to  training 
perfonnance  and  outcomes  in  one  measure  provides  the  ability  to  examine  their  relationship  and 
provide  more  accurate  and  enhanced  feedback.  For  example,  training  aptitude  could  be 
distinguished  from  intangibles  in  instances  where  a  Solider  does  possess  the  intangibles  but  lacks 
the  basic  knowledge,  skills,  and  abilities  to  perform  training.  Or  it  could  be  the  case  that  Soldiers 
need  to  possess  the  requisite  knowledge,  skills,  and  abilities  in  order  to  demonstrate  the 
intangibles. 
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Review  of  Training  Methods  and  Learning  Theories 
Outcome  Based  Training  and  Education  (OBT&E) 

Introduction  to  OBT&E.  The  OBT&E  is  actually  a  philosophy  of  training  and 
education  rather  than  a  specific  training  method  or  learning  theory.  It  has  a  learner-centered 
focus  and  could  be  argued  to  be  more  holistic  than  traditional  training  and  educational 
approaches,  in  that  it  “enables  the  student  to  master  a  subject,  apply  the  subject  appropriately, 
and  synthesize  it  with  other  knowledge”  (McDaniel,  2009).  Other  unique  aspects  of  OBT&E  are 
discussed  in  the  next  section. 

Outcome-based  training  and  education  has  been  identified  as  a  promising  approach  to 
learning.  The  Joint  Warfighting  Center  (JWFC)  Training  Development  Group  (TDG)  Analysis 
Division  has  stated  its  support  of  OBT&E  as  a  potentially  effective  means  of  obtaining  the  joint 
training  outcome  goal  of  developing  “joint  capability-knowledgeable  problem  solvers  and  those 
willing  and  able  to  take  initiative”  (JWFC,  2009,  p.  1).  Recent  work  by  the  Army’s  Asymmetric 
Warfare  Group  (AWG)  suggests  that  outcome  based  training  is  needed  for  today’s  changing 
environment  (Ferguson,  2008).  The  Army  has  begun  to  embrace  these  techniques  institutionally 
with  the  incorporation  of  OBT&E  into  the  Anny  Reconnaissance  Course  (ARC)  at  Fort  Knox 
(Perry  &  McEnery,  2009),  the  Infantry  Captains  Career  Course  (ICCC)  at  Fort  Benning 
(Haskins,  2006),  and  the  Initial  Entry  Training  (IET)  programs  at  both  Fort  Benning  and  Fort 
Jackson  (Tice,  2008). 

Distinguishing  features.  The  primary  distinguishing  feature  of  OBT&E  is  implied  by  its 
name.  The  effectiveness  of  training  in  OBT&E  is  assessed  in  tenns  of  outcomes,  whereas  much 
of  current  training  in  the  Anny  is  assessed  in  tenns  of  meeting  a  standard.  Proponents  of 
OBT&E  argue  that  this  allows  for  improved  perfonnance  over  standards-based  training 
(McDaniel,  2009).  One  argument  is  that  when  given  a  standard  as  a  training  goal,  students  are 
unlikely  to  exceed  that  standard;  they  train  until  just  meeting  the  standard.  The  OBT&E,  though, 
encourages  mastery  because  no  minimum  standard  is  specified. 

Another  distinguishing  feature  of  OBT&E  is  that  it  facilitates  mastery  and  generalization 
of  skills  and  knowledge  attained.  Typically  in  OBT&E,  learners  master  fundamental  skills  and 
principles  in  a  low-stress  environment  then  practice  the  skills  and  apply  the  knowledge  in 
increasingly  stressful,  complex  environments.  Emphasis  is  placed  on  learning  principles,  which 
then  pennits  the  learner  to  adapt  their  skills  and  apply  them  to  a  variety  of  situations,  not  just  the 
situations  they  experienced  in  training.  Thus,  some  argue,  OBT&E  “promotes  the  development 
of  adaptive  thinking,  individual  initiative,  collective  agility,  and  most  importantly,  confidence...” 
(McDaniel,  2009). 

The  OBT&E  requires  training  developers  to  take  a  broader  view  of  the  goals  of  training, 
in  contrast  to  standards-based  training.  Training  developers  must  identify  the  objectives  of  the 
training  in  terms  of  holistic  outcomes  instead  of  discrete  standards.  This  is  not  to  say  that 
standards  have  no  place  in  OBT&E;  standards  can  still  be  used,  but  they  are  not  used  as  the  sole 
measure  of  perfonnance. 


9 


While  not  a  learning  theory,  as  pointed  out  above,  OBT&E  is  an  approach  to  learning  and 
is  therefore  related  to  learning  theories.  This  relationship  is  discussed  in  the  next  section. 

Relationship  to  other  learning  theories.  As  a  philosophy  of  training  and  education, 
OBT&E  provides  the  context  in  which  other  learning  theories  can  be  operationalized  and 
training  methods  can  be  used.  Thus,  OBT&E  is  conducive  to  many  learning  theories  and 
training  methods  with  the  requirement  of  measuring  performance  in  tenns  of  outcomes.  For 
example,  cognitive  learning  theories  can  be  used  in  conjunction  with  a  OBT&E  approach  as  long 
as  performance  is  measured  by  outcomes  rather  than  meeting  standards.  Specific  learning 
theories  including  cognitive  learning  theory,  which  we  may  use  within  the  context  of  OBT&E, 
are  discussed  below. 


Learning  Theories 

To  discern  the  best  approaches  to  training  intangible  skills,  it  is  important  to  understand 
the  approaches  to  learning  that  may  be  most  appropriate  for  these  types  of  skills.  The  complex 
nature  of  intangible  skills  requires  the  learner  to  process  information  about  the  skill  and  how  it 
should  be  used  in  appropriate  situations.  Drawing  upon  theories  of  advanced  learning  can 
therefore  help  better  design  training  that  will  facilitate  learning.  The  learning  theories  that  were 
reviewed  are  classified  as  learner-centered  approaches.  Also,  the  theories  have  been  linked  to 
deep  learning  (Cumow,  et  ah,  2006). 

A  learner  centered  approach  is  defined  as  an  approach  to  learning  that  is  consistent  with 
the  14  learner-centered  principles.  The  14  learner-centered  principles  were  identified  for  a  1990 
Presidential  task  force,  revised  in  1997,  and  prepared  by  the  American  Psychological 
Association’s  (APA)  Board  of  Educational  Affairs  (BEA).  The  14  principles  “focus  on 
psychological  factors  that  are  primarily  internal  to  and  under  the  control  of  the  learner  rather  than 
conditioned  habits  or  physiological  factors.  However,  the  principles  also  attempt  to 
acknowledge  external  environment  or  contextual  factors  that  interact  with  these  internal  factors” 
(APABEA,  1997,  p.  3).  The  14  learner  centered  principles  are  provided  below. 

1 .  Nature  of  the  learning  process  -  The  learning  of  complex  subject  matter  is 
most  effective  when  it  is  an  intentional  process  of  constructing  meaning  from 
information  and  experience. 

2.  Goals  of  the  learning  process  -  The  successful  learner,  over  time  and  with 
support  and  instructional  guidance,  can  create  meaningful,  coherent 
representations  of  knowledge. 

3.  Construction  of  knowledge  -  The  successful  learner  can  link  new  infonnation 
with  existing  knowledge  in  meaningful  ways. 

4.  Strategic  thinking  -  The  successful  learner  can  create  and  use  a  repertoire  of 
thinking  and  reasoning  strategies  to  achieve  complex  learning  goals. 

5.  Thinking  about  thinking  -  Higher  order  strategies  for  selecting  and  monitoring 
mental  operations  facilitate  creative  and  critical  thinking. 

6.  Context  of  learning  -  Learning  is  influenced  by  environmental  factors, 
including  culture,  technology,  and  instructional  practices. 
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7.  Motivational  and  emotional  influences  on  learning  -  What  and  how  much  is 
learned  is  influenced  by  the  motivation.  Motivation  to  learn,  in  turn,  is 
influenced  by  the  individual's  emotional  states,  beliefs,  interests  and  goals, 
and  habits  of  thinking. 

8.  Intrinsic  motivation  to  leam  -  The  learner's  creativity,  higher  order  thinking, 
and  natural  curiosity  all  contribute  to  motivation  to  leam.  Intrinsic  motivation 
is  stimulated  by  tasks  of  optimal  novelty  and  difficulty,  relevant  to  personal 
interests,  and  providing  for  personal  choice  and  control. 

9.  Effects  of  motivation  on  effort  -  Acquisition  of  complex  knowledge  and  skills 
requires  extended  learner  effort  and  guided  practice.  Without  learners' 
motivation  to  leam,  the  willingness  to  exert  this  effort  is  unlikely  without 
coercion. 

10.  Developmental  influences  on  learning  -  As  individuals  develop,  there  are 
different  opportunities  and  constraints  for  learning.  Learning  is  most  effective 
when  differential  development  within  and  across  physical,  intellectual, 
emotional,  and  social  domains  is  taken  into  account. 

1 1 .  Social  influences  on  learning  -  Learning  is  influenced  by  social  interactions, 
interpersonal  relations,  and  communication  with  others. 

12.  Individual  differences  in  learning  -  Learners  have  different  strategies, 
approaches,  and  capabilities  for  learning  that  are  a  function  of  prior 
experience  and  heredity. 

13.  Learning  and  diversity  -  Learning  is  most  effective  when  differences  in 
learners'  linguistic,  cultural,  and  social  backgrounds  are  taken  into  account. 

14.  Standards  and  assessment  -  Setting  appropriately  high  and  challenging 
standards  and  assessing  the  learner  as  well  as  learning  progress  —  including 
diagnostic,  process,  and  outcome  assessment  —  are  integral  parts  of  the 
learning  process.  (APABEA,  1997,  p.4-7). 

Deep  learning  involves  the  acquisition  of  higher  order  skills  through  the  relation  of  new 
concepts  to  existing  experience,  the  distinguishing  of  new  ideas  from  existing  knowledge,  and 
the  critical  evaluation  and  determination  of  key  themes  and  concepts  (Draper,  2009;  Entwistle  & 
Ramsden,  1983;  Pink,  2003;  Ramsden,  1992).  Deep  learning  is  a  relevant  concept  for  intangible 
skill  development  as  it  engages  students  on  multiple  levels  in  order  to  create  knowledge  that  is 
meaningful  and  applicable  to  real  world  situations  (Pink,  2003).  Given  the  complex  nature  of 
intangible  skills,  having  a  deep  understanding  of  when  and  how  to  use  such  skills  is  critical. 

Deep  learning  also  serves  as  the  foundation  for  other  aspects  necessary  for  learning  complex 
skills,  such  as  developing  effective  knowledge  stmctures  and  enabling  the  transfer  of  training  to 
operational  environments  (Lord,  Smith,  Weissbein,  Gully,  &  Salas,  1998). 

There  are  many  theories  available  that  can  provide  a  means  for  understanding  how  to  best 
train  the  intangible  skills.  Specifically,  advanced  learning  theories  can  provide  a  key  foundation 
for  developing  this  training  as  they  target  the  development  of  advanced  or  complex  skills 
(Cumow,  et  al.,  2006).  We  conducted  an  extensive  review  of  advanced  learning  theories  from 
the  training,  education,  and  cognition  literatures  relevant  to  the  development  of  learning  and 
intangible  skills.  These  resulting  set  of  theories  includes: 
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•  Cognitive  learning  strategies,  metacognitive  strategies,  scaffolding,  and  cognitive 

tutoring; 

•  Collaborative  learning  and  virtual  learning  groups; 

•  Constructivism/leamer-centered  education; 

•  Experiential  activities; 

•  Learning  to  learn/self-regulation  training; 

•  Self-explanation  strategy  training;  and 

•  Cognitive  flexibility  theory. 

While  each  of  these  theories  takes  a  slightly  different  approach  to  learning,  the 
centralized  theme  around  each  is  the  emphasis  on  learner-centered  education.  These  theories  are 
not  mutually  exclusive  of  one  another,  but  instead  many  overlap  in  tenns  of  the  specific  issues 
they  address.  This  can  be  beneficial  from  a  training  perspective  in  that  the  theories  can  be 
combined  to  address  issues  relevant  to  intangible  skill  development,  including  knowledge 
building,  learning  through  experience,  and  interacting  with  other  learners.  The  following 
provides  a  brief  summary  of  each  theory  and  how  it  may  be  applicable  to  the  training  of 
intangible  skills. 

Cognitive  Learning  Strategies,  Metacognitive  Strategies,  Scaffolding,  and  Cognitive 
Tutoring.  Given  the  complex  nature  of  intangible  skills,  it  may  be  necessary  to  provide  learning 
support  systems  to  individuals  trying  to  master  them.  These  types  of  systems  are  defined  as 
those  that  provide  tools,  aids,  and  other  guidance  in  order  to  offer  a  sense  of  structure  meant  to 
foster  learning.  The  purpose  of  cognitive  and  metacognitive  learning  strategies,  scaffolding,  and 
cognitive  tutoring  is  to  do  exactly  this,  albeit  through  slightly  different  approaches  (Curnow,  et 
ah,  2006).  Learning  strategies  focuses  on  how  individuals  can  best  process  infonnation,  whereas 
scaffolding  and  cognitive  tutoring  describe  tools  to  support  learning.  Specifically,  cognitive 
learning  strategies  provide  assistance  in  helping  students  acquire,  attain,  retain,  and  retrieve 
infonnation  (Van  Dijk  &  Kintsch,  1983).  These  strategies  include  approaches  such  as  rehearsing 
information  to  be  learned,  using  mnemonic  devices  to  organize  and  store  information,  or 
constructing  mental  representations  (Pressley  &  Woloshyn,  1995).  Metacognitive  learning 
strategies  are  similar,  but  slightly  different  in  that  they  focus  on  teaching  students  how  to  leam 
(Bell  &  Kozlowski,  2010).  These  strategies  involve  developing  students’  ability  to  self-regulate 
and  self-monitor  their  thoughts,  actions,  and  processing  of  infonnation  in  order  to  maximize 
learning  opportunities  (Parush,  Hamm,  &  Shtub,  2002). 

Other  learning  tools  that  can  support  knowledge  and  skill  acquisition  include  scaffolding 
and  cognitive  tutoring.  Scaffolding  involves  providing  a  temporary  support  system  to  the 
learner,  with  pieces  of  the  support  being  removed  as  the  learner  becomes  more  advanced 
(Cuevas,  Liore,  &  Oser,  2002).  This  could  involve  instructors  providing  more  hands  on  guidance 
early  on  in  a  course  and  then  slowly  removing  this  support,  or  it  could  be  some  other  support 
mechanism  that  is  later  removed.  The  final  tool,  cognitive  tutoring,  offers  a  fonn  of  learning 
through  guidance.  Tutors,  whether  they  are  human  instructors  or  technology  based  tutoring 
systems,  provide  guidance  such  that  learners  receive  feedback  to  let  them  know  how  they  are 
doing  throughout  the  learning  process  (Merrill,  Riser,  Merrill,  &  Lands,  1995).  Tutors  already 
have  a  mastery  of  the  knowledge  domain  and  are  therefore  able  to  recognize  when  the  learner 
has  made  or  is  about  to  make  an  error. 
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Together,  these  four  different  types  of  learning  tools  can  provide  a  much  needed  support 
system  by  which  students  can  gain  a  clearer  understanding  and  practical  use  of  complex  skills. 
They  have  all  been  linked  empirically  to  improvements  in  learning  and  development,  especially 
for  deep  learning.  Furthermore,  combining  these  different  tools  can  be  of  great  benefit.  For 
example,  Ringenberg  and  VanLehn  (2006)  explored  the  use  of  an  intelligent  tutoring  system  that 
incorporated  elements  of  scaffolding  to  develop  problem  solving  skills  in  students.  They  found 
that  this  approach  significantly  improved  the  deep  learning  of  problem  solving  skills.  Therefore, 
it  is  expected  that  these  types  of  tools  may  also  enhance  the  learning  of  intangible  skills. 

Scaffolding  and  cognitive  tutoring  are  most  useful  as  methods  for  training  intangible 
skills  during  difficult  and  challenging  training  events.  The  similarity  between  scaffolding  and 
cognitive  tutoring  is  the  use  of  an  instructor,  tutor,  or  knowledgeable  support  system. 

Scaffolding  would  be  an  ideal  training  method  for  getting  trainers  to  take  Soldiers  to  the  point 
where  they  have  reached  their  limits  and  then  move  them  beyond  those  limits.  Trainers  could 
provide  a  scaffold  to  get  Soldiers  to  their  maximum  levels  of  initiative,  will,  grit,  and  hardiness. 
Further,  trainers  could  develop  Soldiers  by  either  removing  support  or  pushing  Soldiers  to  even 
higher  levels  of  achievement  concerning  the  intangible  skills. 

Collaborative  Learning  and  Virtual  Learning  Groups.  Collaborative  learning 
environments  are  those  where  individuals  work  together  to  make  sense  of  a  concept  (McCarthey 
&  McMahon,  1992).  The  basis  of  collaborative  learning  is  built  upon  the  idea  that  learning  is  in 
fact  a  social  process  whereby  individuals  can  learn  through  working  with  one  another  (Bonk  & 
Cunningham,  1998).  During  collaborative  learning,  students  work  together,  share  information 
and  opinions  on  the  topic  being  learned,  clarify  concepts,  and  work  towards  joint  problem 
solving.  This  interaction  helps  learners  develop  a  deeper  understanding  of  a  concept  in  that  it 
provides  an  opportunity  for  them  to  build  knowledge  based  not  only  on  their  own 
comprehension,  but  also  through  the  additive  knowledge  of  others  (Shuffler  &  Goodwin,  2008). 

Given  the  recent  rise  in  technology  and  technology-based  learning  systems,  many  of 
these  collaborative  learning  activities  are  now  carried  out  through  virtual  means  (Orvis,  Wisher, 
Bonk,  &  Olson,  2002).  Although  these  groups  can  be  challenging  due  to  a  lowered  degree  of 
social  presence  (Avolio  &  Kahai,  2003),  they  also  offer  the  key  advantage  of  allowing  learners  to 
participate  in  collaborative  learning  experiences  across  space  and  time.  Furthennore,  there  have 
been  many  instructional  tools  developed  to  reduce  the  potential  problems  with  online  learning, 
such  as  virtual  learning  environments  that  enable  synchronous  (e.g.,  instant  messaging  and 
teleconferencing)  and  asynchronous  (e.g.,  email  and  discussion  boards)  information  exchange, 
provide  opportunities  for  practice  and  feedback,  and  facilitate  social  interactions  (Mueller  & 
Strohmeier,  2010).  Given  the  already  demanding  pace  of  training  for  Soldiers,  providing 
opportunities  to  participate  in  virtual  learning  groups  may  allow  for  the  deeper  learning  of  skills 
in  a  convenient  setting  that  is  accessible  at  all  times  and  locations. 

Collaborative  learning  is  suited  for  the  development  of  intangible  skills  because  Soldiers’ 
initiative,  will,  grit,  and  hardiness  are  likely  to  be  tested  in  collaborative  learning  environments 
such  as  field  exercises  or  combat  training  center  (CTC).  The  communication  and  shared  learning 
experience  that  comes  from  collaborative  learning  environments  would  help  Soldiers  to  develop 
intangible  skills.  Soldiers  would  learn  how  to  build  upon  their  individual  abilities  from  their 
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fellow  Soldiers  through  the  shared  hardships  experienced  in  collaborative  training.  Further, 
Soldiers  could  learn  how  their  demonstration  of  intangible  skills  can  influence  and  affect  their 
fellow  Soldiers.  Virtual  learning  tools  such  as  the  use  of  mobile  devices  could  enhance  the 
communication  and  shared  experience  of  Soldiers  in  complex  and  challenging  collaborative 
training  environments. 

Constructivism/Learner-Centered  Education.  According  to  constructivist  and  learner- 
centered  models  of  instruction,  learners  are  more  apt  to  gain  knowledge  and  skills  when  they 
have  some  degree  of  control  over  the  learning  experience  (Stefanov,  Stoyanov,  &  Nikolov, 

1998).  Traditional  training  approaches  tend  to  treat  learners  as  passive  recipients  of  infonnation 
where  proceduralized  knowledge  acquisition  is  the  goal  (Bell  &  Kozlowski,  2010).  However, 
learner-centered  models  shift  the  focus  to  the  learner  in  order  to  ensure  that  the  learner  is  actively 
involved  in  the  process.  This  enables  the  learner  to  construct  knowledge  over  time  in  a  way  that 
becomes  easy  to  recall.  Surgeons  are  often  used  as  an  example  of  constructive  learning  in  that 
they  gain  a  foundational  knowledge  in  the  classroom  upon  which  they  then  must  draw  from  their 
experiences  to  supplement  and  deepen  their  knowledge  base  (Abernathy  &  Hamm,  1995). 
Eventually,  they  become  experts  because  they  utilize  many  different  techniques  and  approaches 
to  learning  information  that  best  fits  their  needs;  with  each  new  experience  providing  additional 
knowledge. 

One  example  of  this  type  of  learner-centered  model  is  active  learning  (Kozlowski, 
Mullins,  Weissbein,  Brown,  &  Bell,  2001).  This  approach  is  designed  to  stimulate  and  shape 
different  cognitive,  emotional,  motivational,  and  emotion  self-regulatory  processes  such  that 
learners  can  effectively  focus  attention,  direct  effort,  and  manage  emotions  during  learning 
experiences.  This  approach  differs  from  traditional  perspectives  of  training  in  that  it  focuses  on 
the  internal  versus  the  external  regulation  of  learning  (Iran-Nejad,  1990).  This  means  that  active 
learning  puts  the  responsibility  upon  the  learner  to  engage  in  the  learning  process  and  make 
learning  decisions.  The  benefit  of  active  learning  is  that  it  is  especially  helpful  for  developing 
complex  skills  that  are  not  easily  trained  such  as  adaptability.  Indeed,  recent  empirical  research 
has  found  that  the  incorporation  of  active  learning  into  training  designs  has  resulted  in  a  deep 
level  understanding  of  adaptive  expertise  as  well  as  adaptive  transfer  of  different  types  of  skills 
(Bell  &  Kozlowski,  2008;  DeRouin,  Fritzsche,  &  Salas,  2005;  Ivancic  &  Hesketh,  2000).  Given 
this  success  in  training  other  intangible  skills,  it  is  likely  that  active  learning  may  also  be  of 
benefit  to  the  intangible  skills  presented  here. 

Soldiers’  participation  in  field  exercises  and  NTC  training  is  active,  meaning  that 
Soldiers  are  fully  in  control  of  their  behavior  in  a  simulated  exercise  or  problem.  Thus,  when 
Soldiers  are  required  to  call  upon  their  grit,  will,  hardiness,  and  initiative  in  various  simulated 
environments  they  can  practice  utilizing  different  techniques  and  approaches  that  will  give  them 
expert  access  to  the  intangible  skills  in  real  life  situations. 

Experiential  Activities.  Experiential  learning  opportunities  were  highlighted  previously 
as  one  of  the  approaches  recommended  during  the  Phase  I  focus  groups.  From  a  theoretical 
perspective,  experiential  activities  are  those  activities  which  individuals  gain  knowledge  through 
active  reflection  of  experiences.  Kolb  (1984)  posited  that  experiential  activities  stimulate 
knowledge  gain  and  learning  through  the  refinement  of  existing  concepts  and  fonnation  of  new 
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ones  via  sense-making.  He  argued  that  the  basic  assumptions  of  experiential  learning  were  that 
learning:  1)  is  a  process,  not  an  outcome;  2)  derives  from  experience;  3)  requires  an  individual 
to  resolve  dialectically  opposed  demands;  4)  is  holistic  and  integrative;  5)  requires  interplay 
between  a  person  and  environment;  and  6)  results  in  knowledge  creation.  Experiential  learning 
can  be  simulation  based  in  a  specific  training  setting,  or  it  can  be  through  on-the-job  experiences. 
The  benefit  of  this  type  of  learning  is  that  it  can  occur  constantly  throughout  the  learner’s  career 
as  long  as  individuals  take  time  to  reflect  upon  their  experiences  (McCauley,  Moxley,  &  Van 
Velsor,  1998).  This  may  prove  to  be  a  particularly  useful  approach  to  developing  intangible 
skills  for  Soldiers  as  it  does  not  require  much  additional  time  in  the  training  cycle,  but  instead 
can  be  incorporated  throughout  everyday  experiences. 

Learning  to  Learn/Self-Regulation  Training.  Learning  to  learn,  or  self-regulated 
learning,  is  the  process  by  which  learners  take  charge  of  identifying  and  correcting  learning 
problems  (Wisher  &  Graesser,  2007).  This  is  a  metacognitive  approach  in  that  it  involves  the 
higher  level  act  of  recognizing  fluctuations  in  one’s  own  learning  and  reconciling  these 
differences  in  order  to  maximize  a  learning  experience  (Kanfer  &  Heggestad,  1999).  Learning  to 
leam  is  built  upon  the  underlying  belief  that  individuals  are  able  to  leam  how  to  regulate  their 
own  cognition,  motivation,  or  behavior  (Bell  &  Kozlowski,  2008).  Furthermore,  through  these 
processes  of  regulation,  learners  should  be  able  to  achieve  educational  or  developmental  goals 
(Zimmerman,  1989).  Training  that  involves  self-regulation  or  learning  to  leam  not  only 
addresses  learning  objectives  tied  to  skill  development  but  also  incorporates  learning  strategies, 
structure  of  knowledge  and  application  of  knowledge  and  learning  strategies  as  outcomes. 

Self-regulated  learning  is  a  reflective  process  which  may  be  challenging  for  some 
learners  to  develop.  Indeed,  empirical  research  has  resulted  in  mixed  findings  regarding  the 
effectiveness  of  fonnal  attempts  to  train  self-regulation  (Hattie,  Biggs,  &  Purdie,  1996;  Hofer  & 
Yu,  2003;  Simpson,  Hynd,  Nist,  &  Burrell,  1997).  However,  it  has  been  found  to  be  effective  for 
ensuring  adaptive  transfer  (Bell  &  Kozlowski,  2008)  and  improving  emotion  control  (Keith  & 
Frese,  2005).  Furthermore,  self-regulation  training  tends  to  work  best  when  it  combines  teaching 
the  range  of  cognitive  components  of  learning  with  a  range  of  motivational  components  (Hofer 
&  Yu,  2003).  From  the  perspective  of  training  intangible  skills,  incorporating  self-regulation  to 
improve  emotion  control  during  the  training  process  may  be  beneficial  as  trying  to  become 
proficient  at  these  complex  skills  may  lead  to  frustrations.  The  measurement  of  intangible  skills 
will  help  Soldiers  to  identify  developmental  needs  to  learn  how  to  regulate  their  motivation  and 
behavior.  Following  an  administration  of  the  measures,  the  awareness  of  developmental  needs 
can  be  used  to  regulate  and  modify  behavior  in  future  training  events  and  other  life  experiences. 

Self-Explanation  Strategy  Training.  The  premise  behind  self-explanation  strategy 
training  is  that  explaining  examples  is  a  critical  aspect  of  the  learning  process  (VanLehn,  Jones, 

&  Chi,  1992).  In  this  type  of  training,  individuals  are  taught  to  explain  examples  to  themselves 
while  processing  the  content  to  be  learned.  Research  has  illustrated  that  those  who  use  this 
technique  tend  to  have  higher  perfonnance  when  asked  to  apply  the  knowledge  learned. 
Furthennore,  the  use  of  self-explanation  strategy  training  to  support  learning  has  been  effective 
both  when  learners  are  directly  applying  the  rules  they  have  acquired  (i.e.,  near  transfer),  as  well 
as  when  rules  are  applied  more  flexibly  (i.e.,  far  transfer;  Atkinson,  Renkl,  &  Merrill,  2003). 
There  are  many  ways  in  which  learners  can  walk  through  examples.  This  can  include  having 
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students  take  notes  (Trafton  &  Trickett,  2001)  or  explaining  out  loud  their  though  processes 
(VanLehn  et  ah,  1992).  It  can  also  include  having  tutors  initiate  questions  that  students  can  use 
as  a  starting  point  (Chi,  1996)  as  well  as  having  students  justify  the  steps  they  took  in  solving 
example  problems  (Atkinson  et  ah,  2003). 

One  possible  method  of  self-explanation  that  may  be  particularly  useful  for  training 
intangible  skills  is  the  “what  if’  method,  also  known  as  the  “crystal  ball”  technique  (Cohen, 
1998).  This  is  an  explanatory  approach  where  learners  walk  through  conclusions  they  have 
drawn  about  an  event.  Individuals  must  first  imagine  that  the  conclusion  they  have  drawn  and 
believe  to  be  correct  is  actually  wrong.  They  must  then  reflect  on  why  it  might  be  wrong  and 
how  they  may  need  to  adjust  their  thinking  processes  to  account  for  all  possible  circumstances. 
This  type  of  self-explanation  has  been  successfully  used  to  develop  critical  thinking  skills  in 
Soldiers,  and,  therefore,  may  have  relevance  with  other  complex  skills  (Cohen,  et  al.,  1998). 

A  self-explanation  strategy,  particularly  the  “what  if’  method,  is  an  approach  that 
Soldiers  could  use  to  examine  and  help  them  explain  their  own  behavior  concerning  the 
intangible  skills.  This  approach  could  be  introduced  to  Soldiers  following  an  event  that  required 
demonstration  of  the  intangible  skills.  Soldiers  could  be  asked  to  reflect  on  what  went  well  and 
the  conclusions  that  can  be  drawn  with  focus  on  the  intangible  skills. 

Cognitive  Flexibility  Theory.  The  concept  of  cognitive  flexibility  theory  (CFT)  is 
centered  upon  the  idea  that  individuals  may  gain  extensive  knowledge  during  training,  but  they 
also  need  to  be  able  to  apply  this  knowledge  in  situations  that  may  be  very  different  from  those 
experienced  in  training  (Bell  &  Kozlowski,  2010;  Spiro,  Feltovich,  Jacobson,  &  Coulson,  1992). 
In  traditional  approaches  to  training,  individuals  leam  information  in  a  relatively  linear  fashion, 
organized  into  logical  categories  or  frameworks.  However,  it  may  not  always  be  the  case  that  a 
learner  will  need  to  recall  this  infonnation  in  the  exact  same  way.  Cognitive  flexibility  theory 
emphasizes  the  conceptual  interrelatedness  of  different  ideas  and  how  they  may  be 
interconnected  (Jonassen,  2007). 

According  to  this  approach,  knowledge  may  need  to  be  stored  and  retrieved  in  ill-defined 
domains  where  multiple  perspectives  or  schemas  may  be  involved.  For  example,  in  medicine 
there  may  be  many  different  ways  to  treat  a  patient  for  an  illness;  medical  students  must 
therefore  be  prepared  to  understand  and  react  to  the  fact  that  there  is  not  always  a  single  “right” 
answer.  Thus,  a  key  guiding  principle  of  cognitive  flexibility  theory  is  that  information  may 
need  to  come  from  multiple  sources  and  perspectives  in  order  to  ensure  that  learners  will  be  able 
to  think  flexibly  when  required  (Spiro,  Feltovich,  &  Coulson,  2004).  Given  that  intangible  skills 
are  those  that  may  be  applied  in  a  wide  range  of  situations  and  complex  circumstances  beyond 
the  training  environment,  cognitive  flexibility  appears  to  be  a  critical  element  to  the  training 
process.  Soldiers  should  be  introduced  to  a  variety  of  challenging  and  complex  training 
environments  to  increase  their  cognitive  flexibility  concerning  the  intangible  skills.  During  these 
training  events,  the  measurement  of  intangible  skills  will  create  a  source  of  infonnation  that 
Soldiers  can  leam  from  to  improve  performance  in  real  life  situations. 

Learning  Theory  Summary.  The  learner  centered  theories  are  applicable  to  the 
implementation  strategy  for  the  measures.  Each  of  the  learning  theories  discussed  provides 
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specific  strategies  that  will  increase  the  effectiveness  of  intangible  measures.  Based  on  the 
measures  that  are  developed,  multiple  learning  theories  can  be  applied  within  an  OBT&E  context 
to  increase  the  effectiveness  of  training. 

Literature  Review  Summary 

This  report  examined  the  literature  and  existing  data  to  guide  the  design  of  intangible 
measures  and  a  strategy  for  their  implementation.  Concerning  the  design  of  intangible  measures, 
we  examined  the  literature,  relevant  doctrine,  and  existing  measures  to  identify  reliable  and  valid 
measures  and  provide  potential  recommendations  for  improving  upon  them.  Regarding  the 
development  of  a  strategy  for  measurement  implementation,  data  collected  in  Phase  I  was 
reviewed  and  literature  on  training  and  learning  theories  were  examined  to  identify  useful 
training  methods.  The  summaries  for  these  topics  are  below. 

Intangible  Measurement 

The  literature  provided  examples  of  reliable  and  valid  measures  for  all  of  the  intangibles 
except  will.  The  doctrine  on  will  and  the  literature  on  Warrior  Ethos  provided  examples  of 
relevant  content  such  as  perseverance  and  the  ability  to  adapt.  This  literature  serves  as  an 
adequate  foundation  for  the  design  of  a  content  valid  measure  of  will. 

Several  measures  of  initiative  were  identified  (e.g.,  supervisory,  self-report,  and  SJT). 
Research  demonstrated  that  a  supervisor’s  measure  of  employee  initiative  and  an  SJT  measure 
were  most  effective  at  accurately  assessing  initiative.  However,  these  measures  of  initiative 
were  not  developed  for  Anny  personnel.  Therefore,  these  instruments  must  be  adapted  for  Army 
use  by  adding  Anny-specific  language  and  additional  questions  where  necessary.  It  is  beneficial 
that  the  literature  and  its  supporting  validation  research  provide  for  multiple  measures  of 
initiative. 

There  is  one  measure  of  grit  with  two  versions  (i.e.,  short  and  long)  that  are  both  reliable 
and  valid.  These  measures  of  grit  incorporated  two  subscales:  consistency  of  interest  and 
perseverance  of  effort.  Both  measures  had  been  researched  in  an  Army  environment  and 
demonstrated  the  ability  to  predict  training  completion  for  West  Point  cadets.  Researchers 
reported  that  the  grit  measure  was  easy  to  use  and  score  and  that  it  was  useful  for  professional 
development.  Researchers  also  suggested  that  the  grit  measures  be  used  alongside  other 
instruments.  Therefore,  the  grit  measure  may  be  a  good  candidate  for  combining  its 
measurement  with  that  of  other  intangibles  (e.g.,  will).  Further,  this  research  demonstrates  the 
ability  of  the  grit  measures  to  accurately  measure  grit  as  part  of  a  battery  of  measures. 

The  literature  on  hardiness  suggests  that  it  is  a  personality  trait  or  a  set  of  attitudes. 
Therefore,  the  reviewed  measure  of  hardiness  is  typical  of  a  personality  test  (e.g.,  items  that 
assess  attitudes  and  beliefs  rather  than  behaviors).  This  type  of  measure  is  less  applicable  to  a 
field  training  environment  where  the  assessment  of  observable  behaviors  is  preferred.  The 
literature  identified  three  components  of  hardiness:  commitment,  control,  and  challenge.  These 
components  can  be  used  to  design  new  instruments.  However,  questions  remain  concerning  the 
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type  of  test  (e.g.,  personality  or  behavioral)  and  who  would  rate  the  test  (e.g.,  self,  supervisor,  or 
training  support  personnel). 

The  definition  of  initiative  and  its  measures  distinguishes  it  from  the  other  intangibles 
because  of  the  focus  on  self-starting  behaviors.  Hardiness  is  distinct  from  grit  and  will  in  that  it 
is  seen  as  a  set  of  attitudes  or  a  personality  trait.  For  these  reasons,  these  intangibles  may  not  be 
combined  suitably  with  the  other  intangibles,  but  a  measure  could  still  be  designed  for  them 
separately. 

The  intangibles  that  would  seem  most  easily  combined  are  will  and  grit.  The  overlap  in 
definitions  and  the  focus  on  perseverance  suggests  they  are  similar.  The  one  distinction  is  that 
grit  concerns  long-term  achievements  more  than  will.  Existing  grit  measures  have  demonstrated 
reliability  and  validity,  whereas,  no  existing  measures  of  will  were  identified.  Items  could  be 
designed  using  doctrine  on  will  together  with  Warrior  Ethos  and  added  on  to  either  grit  measure 
(i.e.,  short  or  long). 

Training  and  Learning  Methods 

Findings  from  Phase  I  data  are  relevant  to  creating  a  strategy  for  implementation.  The 
high  OPTEMPO  forces  a  training  environment  that  is  constrained  by  resources  and  time.  Thus, 
there  is  limited  time  to  train  the  tangible  skills  and  even  less  time  to  focus  on  training  relevant 
intangibles.  Therefore  intangible  measurement  and  training  should  be  designed  to  occur  during 
other  training  events  (e.g.,  field  exercises  or  gunnery).  Soldiers  suggested  that  training  needs  to 
be  challenging.  Soldiers  also  identified  that  training  is  an  iterative  process  and  if  practice  is  not 
provided  under  normal  conditions,  task  accomplishment  under  more  difficult  training  conditions 
will  be  less  effective. 

The  OBT&E  has  been  indicated  as  an  effective  training  philosophy  for  implementing 
training  on  the  intangibles.  The  OBT&E  is  a  learner  centered  philosophy  and  will  allow  for  the 
incorporation  of  multiple  learning  theories  into  training.  The  important  design  recommendation 
from  the  literature  on  OBT&E  is  that  training  needs  to  focus  on  evaluating  outcomes  rather  than 
standards.  This  is  complemented  by  Phase  I  findings  which  suggest  a  crawl,  walk,  run  approach 
to  training  where  the  run  stage  consists  of  ambiguous,  uncertain,  and  rigorous  training 
environments  that  challenge  Soldiers  to  practice  and  apply  previous  training  to  accomplish  a 
mission  rather  than  achieve  a  standard. 

The  learning  theories  can  be  applied  to  develop  an  effective  implementation  strategy. 

The  learner-centered  approach  of  the  reviewed  learning  theories  is  compatible  with  the 
philosophy  of  OBT&E.  Each  theory  can  be  used  to  build  effective  implementation  features  into 
training.  Which  theories  are  leveraged  depends  upon  the  measures  that  are  developed  and  also 
the  training  resources  that  are  available.  Some  examples  of  how  the  learning  theories  could  be 
applied  to  an  implementation  strategy  are  discussed  below. 

Cognitive  flexibility  theory  and  experiential  training  are  learning  strategies  that  attend  to 
the  stressful,  complex,  and  difficult  operational  environment  faced  by  Soldiers.  These  theories 
emphasize  the  importance  of  highly  realistic  training  environments  where  Soldiers  must  react  to 
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uncertainty  and  apply  their  knowledge,  skills,  and  abilities  in  innovative  ways  to  find  creative 
solutions. 

Learning  theories  such  as  scaffolding,  cognitive  tutoring,  and  collaborative  learning 
emphasize  the  importance  of  a  support  system  to  learning.  These  theories  should  be 
incorporated  into  implementation  strategy  by  identifying  the  support  systems  that  are  readily 
available  during  training.  These  support  systems  could  be  used  to  build  Soldier  confidence  as 
conditions  are  made  more  difficult  and  task/mission  accomplishment  becomes  more  difficult. 

Soldiers  noted  the  importance  of  training  that  pushes  them  to  their  breaking  point. 
Following  training,  self-explanation  theory  may  be  useful  in  getting  Soldiers  to  identify  their 
breaking  point,  why  it  is  there,  and  how  they  can  overcome  it.  This  learning  theory  could  be 
combined  with  other  learning  theories  such  as  collaborative  learning  or  cognitive  tutoring  so  that 
a  Soldier’s  explanation  was  heard  by  a  support  system  capable  of  knowing  when  and  when  not  to 
intervene. 

The  literature  review  of  intangibles  training  methods  and  their  measures  served  to  inform 
the  creation  of  protocols  (questionnaire,  interview,  and  focus  group)  for  this  second  phase  of  data 
collection.  One  outcome  desired  of  this  data  collection  was  to  leam  more  about  how  to 
effectively  implement  measures  of  intangibles  into  BCT  readiness  training.  Another  outcome 
was  to  build  on  the  Phase  II  literature  review  and  identify  performance  indicators  for  the 
intangibles.  The  combination  of  knowledge  gained  from  achieving  both  of  these  data  collection 
outcomes  is  important  to  the  development  of  content  valid  measures  of  select  intangible 
psychological  constructs. 


Method 


Research  Questions 

The  data  collection  sought  answers  to  the  following  research  questions: 

1 .  What  training  events  are  most  effective  in  training  the  intangibles? 

2.  What  training  methods  and  learning  theories  are  used  in  existing  exercises,  venues,  and 
activities? 

3.  What  training  features  are  used  in  existing  exercises,  venues,  and  activities? 

4.  How  can  measurement  of  intangibles  be  accomplished  and  adapted  to  the  field? 

5.  What  measurement  characteristics  are  most  appropriate  for  measuring  intangibles? 

6.  What  are  the  training  performance  indicators  for  the  selected  intangibles? 

Sample 

Data  collection  occurred  at  Fort  Hood  with  Soldiers  from  a  BCT.  Data  collection 
sessions  consisted  of  four  focus  groups  and  14  interviews  resulting  in  a  total  sample  size  of  47 
active  duty  Anny  personnel.  The  job  position  characteristics  of  the  sample  were  selected  to 
provide  representation  from  various  rank  levels  with  backgrounds  in  training  (e.g.,  training  plans, 
design,  execution,  and  experience  conducting  training).  Consequently,  a  focus  group  protocol 
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was  used  to  collect  data  from  separate  groups  of  company  commanders,  first  sergeants,  platoon 
sergeants,  and  platoon  leaders.  And  an  interview  protocol  was  used  to  collect  data  from  battalion 
and  brigade  commanders,  command  sergeant  majors,  operations  officer  (S3),  executive  officers 
(XO),  training  Noncommissioned  Officer  (NCO),  and  operations  NCOs.  Table  1  displays  the 
breakdown  of  the  sample  by  rank  cohort. 

Table  1. 


Sample  Size  of  Interviews  and  Focus  Groups  by  Rank  Cohort 


Rank  Cohort  Sample  Size 

Field  grade  officers  (lieutenant  colonels  and  majors)  8 

Company  grade  officers  (captains  and  lieutenants)  17 

Senior  NCOs  (sergeant  majors,  first  sergeants,  master 
sergeants  and  sergeants  first  class)  17 

Junior  NCOs  (staff  sergeants  and  sergeants)  2 

No  rank  indicated  3 


Procedures 

All  Soldiers  were  first  given  a  Privacy  Act  Statement  and  Informed  Consent  Statement 
before  the  session.  In  this  report,  the  tenn  Soldiers  is  used  to  refer  to  all  participants  (e.g., 
enlisted,  NCOs,  and  officers).  Across  all  data  collection  sessions  no  one  elected  to  not 
participate.  Next,  Soldiers  filled  out  a  questionnaire  (Appendix  A).  The  questionnaire  consisted 
of  items  in  four  different  content  domains:  training  methods,  training  features,  rater  and  time 
concerns,  and  intangibles.  Each  item  on  the  questionnaire  used  a  5  point  scale  (1  =  Strongly 
Disagree  to  5  =  Strongly  Agree)  asking  respondents  to  rate  the  extent  to  which  likely  training 
events  provides  for  a  given  training  method,  training  feature,  rater  and  time  concerns,  and  the 
measurement  of  specific  intangibles.  For  example,  an  item  from  the  training  methods  content 
domain  asked  respondents  the  extent  to  which  they  agreed  or  disagreed  that  a  given  training 
event  (e.g.,  field  exercise,  gunnery,  etc.)  provided  realism.  The  rated  training  events  included: 

•  Field  Exercise; 

•  NTC  (National  Training  Center); 

•  Gunnery; 

•  Resiliency  Training; 

•  Combat  Life-saver  Training/Medical  Simulation  Training  Center  (MSTC) 
Training/Trauma  Lane; 

•  Ranger  School/Mungadai  Training  (physical  fitness  events  combined  with  combat 
readiness  tasks  executed  individually  or  in  small  teams); 

•  Basic  Training  Problem  Solving  Exercises; 

•  Classroom  Training; 

•  Power  Point  Training;  and 

•  On  the  Job  Training. 

The  training  events  listed  above  as  well  as  some  of  the  items  for  each  of  the  questionnaire 
content  domains  was  created  based  on  research  conducted  previously.  For  example,  the  items  in 
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the  training  feature  domain  were  identified  in  Phase  I  of  this  research  as  features  that  are  known 
to  contribute  to  effective  training.  Another  example  of  how  the  questionnaire  was  built  on 
previous  findings  is  that  each  item  in  the  training  method  domain  was  associated  with  its 
respective  learning  theories  identified  from  the  Phase  II  literature  review.  Table  2  displays  the 
association  between  questionnaire  items  and  learning  theories. 

Table  2. 


Questionnaire  Item  and  Learning  Theory  Association 


Item 

Questionnaire  Item 

Learning  Theory 

Source 

1 

provides  practice  and  experience  in  training 

Experiential  activities 

Kolb  (1984) 

2 

provides  realism  during  training 

Experiential  activities 

Kolb  (1984) 

3 

provides  multiple,  different  opportunities  to 
apply  knowledge  and  skills 

Cognitive  flexibility  theory 

Spiro,  Feltovich, 
Jacobson,  & 
Coulson  (1992) 

4 

offers  a  collaborative  or  team-based  training 
experience 

Collaborative  learning  and 
virtual  learning  groups 

McCarthey  & 

McMahon 

(1992) 

5 

allows  Soldiers  to  explain  their  grasp  of  the 
training  exercise  and  discuss  examples  of  their 
performance  during  training 

Self-explanation  strategy 
training 

VanLehn,  Jones, 

&  Chi  (1992) 

6 

provides  a  support  structure  (e.g.,  peers  or  a 
leader)  helpful  in  building  knowledge  of 
simple  tasks  into  complex  skills 

Cognitive  learning 
strategies,  scaffolding,  and 
cognitive  tutoring 

Cuevas,  Fiore,  & 
Oser  (2002) 

7 

has  an  instructor  or  leader  who  is  able  to  guide 
and  challenge  development  during  training 

Cognitive  learning 
strategies,  scaffolding,  and 
cognitive  tutoring 

Merrill,  Riser,  & 
Merrill  (1995) 

8 

involves  understanding,  problem  solving,  and 
making  sense  of  new  events  rather  than  the 
memorization  of  facts 

Constructivism/leamer- 
centered  education 

Bell  & 

Kozlowski 

(2010) 

9 

allows  Soldiers  to  critique  their  performance 
and  assess  their  ability  to  learn  during  training 

Learning  to  learn  training 

Wisher  & 

Graesser  (2007) 

Note.  The  identified  source  is  a  representative  source  and  others  exist. 


After  Soldiers  completed  the  questionnaire,  they  were  asked  a  series  of  open-  ended 
questions  in  either  a  focus  group  or  interview  setting.  Appendix  B  contains  a  complete  list  of  all 
questions  and  probes  employed  during  interview  and  focus  group  sessions.  Interviews  were 
conducted  by  the  interviewer  either  in  person  or  on  the  phone.  Phone  interviews  were  digitally 
recorded  while  in  person  interviews  were  manually  transcribed  by  a  recorder.  Complete 
transcripts  were  created  for  all  interview  and  focus  group  sessions. 
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Data  Analysis 


Quantitative  analysis.  Means  and  standard  deviations  were  calculated  for  all  items  on 
the  questionnaire.  Training  event  mean  scores  were  created  for  each  item.  For  example,  the 
mean  scores  derived  from  rating  the  realism  of  a  given  training  event  were  likewise  calculated 
for  all  training  events  (e.g.,  field  exercises  and  NTC).  By  way  of  a  comparison  of  these  mean 
scores,  it  can  be  detennined  which  training  events  Soldiers  believed  to  be  better  at  providing 
realism,  offering  a  collaborative  environment,  etc.  This  method  of  analysis  was  used  for  each  of 
the  questionnaire  content  domains  (i.e.,  training  methods,  training  features,  rater  and  time 
concerns,  and  intangibles).  Thus,  the  results  demonstrate  which  training  methods  and  training 
features  are  most  appropriate  for  the  various  training  events,  what  rater  and  time  concerns  are 
present  in  each  of  the  training  events,  and  which  training  environment  is  most  effective  for  the 
measurement  of  intangibles.  The  means  and  standard  deviations  for  each  of  the  content  domains 
are  presented  in  Appendices  C,  D,  E,  and  F. 

Qualitative  analysis.  Qualitative  analysis  of  interview  and  focus  group  transcripts 
consisted  of  coding  each  session  for  themes.  There  were  two  desired  outcomes  of  qualitative 
data  analysis:  1)  identify  the  highest  frequency  themes  and  2)  identify  quotes  that  could  provide 
context  or  add  insight  to  the  questionnaire  findings.  To  adequately  capture  the  importance  or 
emphasis  Soldiers  placed  on  a  given  theme,  two  categories  of  themes  were  identified.  Themes 
were  categorized  under  research  questions  and  only  the  most  frequent  themes  are  discussed. 
Similar  to  the  Phase  I,  analysis  of  the  qualitative  interview  and  focus  group  data  was 
accomplished  using  a  three  step  process. 

Step  I.  Facilitators  and  recorders  from  the  data  collection  read  through  each  transcript 
and  identified  a  tentative  list  of  themes.  They  then  came  to  consensus  on  themes  for  each 
research  question.  A  master  list  of  themes  was  created  and  disseminated  to  all  coders.  A  single, 
representative  transcript  was  then  selected  and  every  coder  used  the  master  theme  list  to  code  its 
comments  (see  Appendix  G).  Discrepancies  in  theme  coding  among  coders  were  discussed  and 
issues  were  resolved  prior  to  coding  all  remaining  transcripts.  In  so  doing,  it  was  confirmed  that 
coders  were  consistently  coding  transcript  comments. 

Step  II.  Eighteen  (18)  total  transcripts  (14  interview  transcripts  and  four  focus  group 
transcripts)  were  then  coded  by  two  coders.  Each  transcript  was  coded  twice  which  allowed  for 
coding  accuracy  checks  in  the  Step  III  to  follow.  Coders  coded  themes  at  the  session  level;  when 
a  theme  was  mentioned  once  in  a  session,  it  was  reported  once  in  the  results.  Similarly,  when  a 
theme  was  mentioned  five  times  in  a  session  it  was  reported  only  once  in  the  results.  This 
allowed  for  the  calculation  of  theme  counts  among  sessions  while  controlling  for  method  issues 
that  could  result  from  analyzing  interview  and  focus  group  data  together.  For  example,  this 
method  of  calculating  theme  counts  does  not  give  greater  weight  to  focus  groups  where  multiple 
Soldiers  are  likely  to  mention  the  same  theme.  It  also  mitigates  the  repetitive  mention  of  a  theme 
by  a  Soldier  in  the  same  session.  Consequently,  this  session  level  method  of  calculating  themes 
ensures  the  magnitude  of  themes  is  not  inflated  or  overstated  in  the  results. 

To  facilitate  capturing  in  depth  descriptions  of  Soldier  comments,  each  coder  highlighted 
the  accompanying  narrative  of  a  given  coded  comment.  This  procedure  allowed  analysts  to 
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include  statements  representative  of  a  particular  theme.  Consequently,  themes  are  reported  in  a 
way  that  provides  description  of  its  richness  and  depth. 

Step  III  Each  transcript  was  coded  twice  by  two  different  coders.  Following  coding,  the 
two  coders  discussed  the  themes  they  identified  and  the  respective  text  from  the  transcripts  that 
they  highlighted.  A  total  of  two  4-hour  accuracy  sessions  took  place  among  coders  where  they 
reviewed  the  transcript  that  they  had  both  coded.  During  each  session,  one  coder  created  a  new 
document  for  each  transcript  that  included  all  of  the  agreed  upon  themes  from  both  coders.  In 
the  accuracy  sessions,  coders  found,  discussed,  and  resolved  differences,  thus  providing  for 
greater  accuracy  throughout  the  coding  process. 

Results  and  Findings 


Intangible  Measurement 

Soldiers  were  asked  to  identify  the  effectiveness  of  each  of  the  different  training  events 
for  measuring  the  intangibles:  initiative,  will,  grit,  and  hardiness.  The  means  of  Soldiers’ 
responses  were  calculated  for  each  of  the  10  training  events  and  for  each  of  the  four  intangibles 
(Appendix  F).  A  sample  item  is  “NTC  is  an  excellent  environment  to  measure  Soldier's 
initiative.”  The  intangibles  were  evaluated  separately.  For  each  intangible,  mean  scores  on  the 
various  training  events  were  ranked  to  determine  the  most  effective  and  least  effective  training 
event. 


The  Soldiers  tended  to  rate  the  same  events  as  effective  at  measuring  each  of  the 
intangibles.  For  example,  the  most  effective  training  events,  from  highest  to  lowest,  across  all  of 
the  intangibles  were  Ranger  School,  the  NTC,  field  exercises,  on-the-job  training  (OJT),  and 
gunnery.  Similarly,  the  least  effective  training  events  for  measuring  intangibles  were  classroom 
training  and  PowerPoint  instruction.  A  frequent  theme  (61%  of  sessions)  provided  insight  into 
the  distinction  between  the  highest  and  lowest  rated  events.  A  quote  representative  of  this  theme 
is  “Baseline  knowledge  is  more  appropriate  for  some  training  events,  and  others  are  more 
appropriate  for  training  complex  tasks,  including  intangibles.”  For  example,  one  Soldier  said  “I 
see  them  (i.e.,  training  events)  as  a  stair  step  approach  to  training  where  you  are  teaching 
individual  skills  and  then  you  throw  in  different  events  of  problem  solving  building  up  to 
collective  training  with  gunnery,  to  field  exercises,  to  NTC  which  is  the  ultimate  and  best 
training  for  a  couple  different  reasons.”  Thus,  Soldiers  seem  to  indicate  that  collective  training, 
involving  a  myriad  of  tasks  and  challenges,  is  a  more  effective  training  environment  for 
measuring  intangibles  than  individual  knowledge  and  skill  acquisition  training  events. 

Learning  Methods 

Soldiers  were  asked  to  identify  their  level  of  agreement  concerning  the  learning  methods 
evident  in  a  given  training  event.  For  example,  “NTC  provides  practice  and  experience  in 
training.”  The  results  below  discuss  Soldiers’  responses  for  each  of  the  learning  methods  and  the 
effectiveness  of  each  training  event  at  providing  them. 
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Experiential  learning.  Soldiers  were  asked  whether  the  various  training  events  provided 
practice  and  experience  in  training.  Results  indicated  that  the  top  three  training  events  for 
providing  practice  and  experience  were  NTC  (M=  4.52,  SD=. 81),  field  exercises  (M=4.47, 

SD=. 66),  and  OJT  (M=  4.35,  SD=.6 4).  Soldiers  frequently  (50%  of  sessions)  mentioned  practice 
and  experience  as  an  effective  training  method.  Soldiers’  comments  in  interview  and  focus 
group  sessions  described  the  effectiveness  of  these  events  and  demonstrated  some  of  the  specific 
methods  used  to  provide  experiential  training. 

Concerning  the  effectiveness  of  field  exercises  at  providing  experiential  training  on  the 
intangibles,  one  Soldier  said  “Putting  a  new  Soldier  in  that  environment  (field  exercises)  for  2 
weeks,  making  them  function,  by  the  end  of  that  they  are  integrated  and  had  a  test  in  all  of  these 
intangibles.  After  two  weeks,  you  know  your  guys  (there  is  nowhere  to  hide).  At  the  end  of  that, 
you  know  individual  strengths  and  weaknesses.”  Another  Soldier  said  “Need  to  put  combat  anns 
guys  out  in  their  element  -  that  is  what  they  joined  the  Army  to  do.  The  only  way  to  learn  to 
walk  through  the  woods  is  to  walk  through  the  woods.  Everything  else  is  sit  around  and  talk 
about  it  -  except  OJT.”  Lastly  one  Soldier  discussed  his  sequential  method  for  providing 
experiences  for  his  Soldiers,  “Put  Soldiers  up  front  and  force  them  to  do  something  -  make  them 
do  the  leader’s  job.”  Another  Soldier  said  “You  start  with  individual  skills,  then  gunnery,  then 
field  exercises,  and  then  NTC  environment.” 

Learning  under  realistic  conditions.  Soldiers  were  asked  whether  the  various  training 
events  provided  for  realistic  training.  The  three  highest  rated  events  in  terms  of  providing 
realism  during  training  were  the  NTC  (M=4.48,  SD=. 78),  Ranger  School/Mungadai  training 
(M=4.20,  SD=. 86),  and  field  exercises  (M= 4. 11,  SD=. 69).  Notably,  the  mean  scores  for  realism 
were  lower  than  the  other  training  methods.  This  may  indicate  a  gap  between  what  Soldiers 
know  real  world  military  missions  are  like  and  how  effective  training  events  are  at  replicating 
that  realism.  Realism  in  training  was  also  the  most  frequently  mentioned  (78%  of  sessions) 
qualitative  theme  from  among  the  training  methods.  In  focus  groups  and  interviews,  Soldiers 
said  that  providing  realism  in  training  was  an  important  and  necessary  training  method.  One 
Soldier  described  the  importance  of  realism  to  the  training  of  basic  skills  “The  realism  with 
regard  to  NTC,  field  exercises,  and  gunnery  -  those  are  core  fundamentals  at  how  we  get  good  at 
our  profession.”  Another  Soldier  mentioned  the  importance  of  realism  and  experiential  training 
prior  to  deployment,  “Mostly  realism  and  practice  experience  in  training.  Because  I  believe  the 
first  time  you  are  tested  should  not  be  in  combat.  You  should  find  out  how  strong  you  are  prior 
to  having  boots  on  the  ground.  If  you  find  out  what  kind  of  man  you  are  before  you  are  tested 
for  real. .  .the  more  real  the  training  that  you  have  and  the  more  you  practice,  the  better  it  would 
be.”  Thus  realistic  training,  training  that  resembles  as  close  as  possible  the  conditions  Soldiers 
will  face  in  executing  their  missions  is  a  very  important  ingredient  to  learning.  That  this  learning 
method  or  condition  was  rated  lower  than  others  may  indicate  improvement  in  its  application  is 
needed. 


Demonstrating  cognitive  flexibility.  Soldiers  were  asked  whether  the  various  training 
events  provided  for  multiple,  different  opportunities  to  apply  knowledge  and  skills.  The  three 
highest  rated  events  were  NTC  (M=4.52,  51)=. 84),  field  exercises  (M=4.38,  SD=4.33)  and 
Ranger  School/Mungadai  training  (M=4.33,  SD=.12).  In  the  interview  and  focus  group  sessions 
this  was  a  frequent  theme  (50%  of  sessions).  Soldiers  cited  this  training  method  in  reference  to 
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NTC  or  field  exercises.  For  example,  one  Soldier  said  “The  other  events  like  field  exercises  etc. 
bring  more  than  one  piece  of  training  -  it  is  variety  and  different  each  time.”  Another  Soldier 
said  that  there  was  value  in  this  training  method  being  used  alongside  repetition  in  training 
“Multiple  opportunities  with  repetition.  The  NTC  and  gunnery  are  what  I’m  familiar  with  -  it 
improves  a  unit  faster;  more  dollar  for  your  investment.” 

Collaborative  learning.  Soldiers  were  asked  whether  the  various  training  events  offer  a 
collaborative  or  team-based  training  experience.  The  three  highest  rated  events  for  offering  a 
collaborative  or  team-based  training  experience  were  NTC  (M=4.63,  SD=. 74),  field  exercises 
(M=4.38,  SD=. 58),  and  Ranger  School/Mungadai  training  (M=4.27,  £0=1.10).  In  focus  group 
sessions,  this  training  method  was  frequently  mentioned  (50%  of  sessions).  One  Soldier  said 
“NTC  was  great  -  like  a  field  exercise  but  more  encompassing  -  had  to  rely  on  others  to  do  their 
job.”  Further,  one  Soldier  described  the  effectiveness  of  NTC  and  field  exercises  at  building 
team  models  “I  like  the  teamwork  part  of  it  -  team  based  training  builds  your  team  models  -  and 
that  is  where  you  create  standard  operating  procedures  (SOP)  and  find  out  who  knows  what  and 
who  you  can  rely  on  -  you  don’t  get  that  back  here  in  garrison  -  but  seeing  it  at  NTC  is  different 
-  also  at  a  field  exercise.”  Referring  to  the  effectiveness  of  Mungadai  training  for  training 
intangibles  one  Soldier  said  “To  develop  intangibles  -  best  I’ve  seen  in  my  career  is  the 
Mungadai  -  they  wear  your  a**  out  for  three  days.  It’s  physically  challenging,  and  pushes  you 
as  a  group.” 

Soldiers  also  frequently  mentioned  (33%  of  sessions)  the  value  of  group  or  team 
feedback  during  After  Action  Reviews  (AAR).  One  Soldier  said  “AARs  are  useful  because 
multiple  perspectives  come  out  -  facilitated  by  a  superior  -  it  allows  units  to  self  discover 
strengths  and  weaknesses.  Superior  needs  to  ask  the  right  questions  to  draw  out  the  discussion.” 
Thus,  team-based  AARs  and  group  feedback  enhance  the  use  of  team  building  and  collaborative 
training  methods. 

Self-explanation.  Soldiers  were  asked  whether  the  various  training  events  allow  them  to 
explain  their  grasp  of  the  training  exercise  and  discuss  examples  of  their  perfonnance  during 
training.  The  three  highest  rated  events  for  this  training  method  were  NTC  (M=4.42,  SD=. 87), 
field  exercises  (M=4.1 1,  £D=.81),  and  OJT  (M=  3.93,  SD=. 65).  However,  self-explanation  was 
not  a  frequently  mentioned  theme  in  the  interview  and  focus  group  sessions. 

Training  events  and  instructor  support.  Soldiers  were  asked  whether  the  various 
training  events  provided  a  support  structure  (e.g.,  peers  or  a  leader)  helpful  in  building 
knowledge  of  simple  tasks  into  complex  skills.  The  three  highest  rated  training  events  were 
NTC  (M=4.30,  £D=.98),  field  exercises  (M=4.25,  SD=  .81),  and  Ranger  School/Mungadai 
training  (M=4.07,  SD=. 96).  Also,  Soldiers  were  asked  whether  the  various  training  events  have 
an  instructor  or  leader  who  is  able  to  guide  and  challenge  development  during  training.  The 
three  highest  rated  training  events  were  Ranger  School/Mungadai  training  (M=4.36,  SD=. 93), 
NTC  (M=4.27,  SD=. 96),  and  the  combat  life-saver  training/MSTC  training/trauma  lane 
(M=4.26,  SD=. 79).  Notably,  these  means  were  highest  in  comparison  to  other  training  methods 
signifying  that  this  training  method  is  prevalent  in  a  number  of  the  training  events.  The 
instructor  supported  training  method  was  a  frequently  mentioned  theme  (44%  of  sessions). 
Soldier  comments  concerning  this  training  method  identified  the  continuous  presence  of  an 
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evaluator  as  an  important  aspect  to  the  success  of  training  events.  For  example,  one  Soldier  said 
“having  a  good  OC  (Observer  Controller)  at  NTC  is  great  for  problem  solving.”  Other  Soldiers 
mentioned  that  at  Ranger  School  and  NTC  an  instructor/OC  is  “always  there.” 

Several  Soldiers  provided  comments  that  demonstrated  the  reasons  behind  why  it  is 
important  to  have  good  instructor  support  during  training.  Referring  to  a  field  exercise,  one 
Soldier  said  “. . .  you  have  the  evaluator  there  that  is  a  coach  on  simple  tasks;  helps  you  build 
proficiency;  it’s  somewhat  simple,  but  you  are  supported  by  the  coach  and  your  crew  starting 
with  simple  tasks  and  moving  up.”  Referring  to  the  NTC,  one  Soldier  said  “Now  it’s  a  little 
more  where  they  will  coach  and  mentor  and  allow  for  growth.  Again,  because  of  that  you  can 
really  see  who  is  there  to  get  something  out  of  it.  Or  who  is  there  to  just  kind  of  go  through  the 
motions  and  are  counting  down  their  days.” 

Soldiers  also  frequently  mentioned  the  importance  of  having  a  role  model  or  a  leader  who 
leads  by  example  during  training  (33%  of  sessions).  For  example,  one  Soldier  said  “Soldiers 
will  follow  their  leaders  -  you  need  a  good  leader  -  if  he  quits,  that  does  not  help  that  younger 
Soldier  out  at  all.  Everyone  grew  up  learning  or  being  mentored  by  someone.  That  is  now 
lacking.  Leadership  is  contagious  -  influencing  positively  or  killing  the  dream.”  Another  Soldier 
identified  a  link  between  leading  by  example  and  hardiness  and  mentioned  the  importance  of 
having  a  superior  who  demonstrates  how  to  appropriately  deal  with  stress  “They  (good  leaders) 
can  deal  with  stress.  Whether  that  means  they  keep  going  in  front  of  Soldiers  and  then  letting  it 
out  in  private.  You  don’t  have  to  keep  it  in  all  the  time  to  be  hard,  you  have  to  keep  it  in  and  use 
an  appropriate  outlet,  e.g.,  behind  closed  doors  in  an  office.  Throwing  a  temper  tantrum  in  a 
large  group  isn’t  hard.”  Thus,  beyond  instructor  support,  unit  leader  support  and  their  role 
modeling  was  a  source  of  Soldier  learning. 

Learner-centered  instruction.  Soldiers  were  asked  whether  the  various  training  events 
involved  understanding,  problem  solving,  and  making  sense  of  new  events  rather  than  the 
memorization  of  facts.  The  three  highest  rated  training  events  were  NTC  (M=4.5 1,  SD=. 92), 
field  exercises  (M=4.36,  SD=.8l),  and  Ranger  School/Mungadai  training  (M=4.20,  SD=. 94). 
This  learning  method  was  not  a  frequently  mentioned  theme  in  the  interview  and  focus  group 
sessions. 

Learning  to  learn.  Soldiers  were  asked  whether  the  various  training  events  allow 
Soldiers  to  critique  their  perfonnance  and  assess  their  ability  to  learn  during  training.  The  three 
highest  rated  training  events  were  NTC  (M=4.43,  SD=. 91),  field  exercises  (M=4.34,  SD=. 83), 
and  gunnery  (M=4.20,  SD=. 76).  This  method  of  learning  was  frequently  mentioned  in  interview 
and  focus  group  sessions  (28%  of  sessions).  One  Soldier  highlighted  the  effectiveness  of  this 
training  method  in  a  field  exercise  saying  “When  I  saw  them  in  field  exercises  I  learned  more; 
they  could  critique  their  own  performance  as  they  do  their  task.”  Another  Soldier  mentioned  the 
effectiveness  of  using  self-explanation  during  an  AAR  “If  they  can  tell  you  what  they  screwed 
up  on  and  if  they  can  identify  the  fix  for  it  -  that  is  good.  If  it’s  all  leader  based  -  telling  them 
how  they  screwed  up  -  then  it’s  not  as  well  done.  But  if  they  can  do  that  themselves,  they  will 
learn  from  it.  Telling  them  it  was  done  wrong  will  not  get  them  there.” 
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Summary.  Three  training  events  were  consistently  rated  high  for  their  providing, 
allowing  for,  or  using  the  various  learning  methods:  the  NTC,  field  exercises,  and  Ranger 
School/Mungadai  training.  Across  all  training  events,  the  most  utilized  learning  method  was 
having  an  instructor  or  leader  who  is  able  to  guide  and  challenge  development  during  training. 
The  least  utilized  learning  method  or  condition  was  providing  realistic  training.  In  many  cases, 
the  Soldiers  identified  multiple  learning  methods  that  could  be  used  for  the  different  events.  This 
suggests  that  as  many  learning  methods  as  resources  will  allow  should  be  applied  to  training 
events.  Also,  those  training  events  that  are  already  rich  with  the  use  of  learning  methods  (e.g., 
the  NTC,  field  exercises,  and  Ranger  School/Mungadai)  are  also  the  same  events  rated  as  highly 
effective  for  the  measurement  of  intangibles.  These  three  events  are  also  the  same  ones  that 
Soldiers  identified  in  focus  groups  and  interviews  as  best  suited  for  the  training  of  intangibles. 
Thus  while  causality  cannot  be  inferred,  it  does  seem  to  follow  that  training  events  possessing  a 
profusion  of  learning  methods  are  the  same  ones  with  the  greatest  potential  for  the  training  and 
measurement  of  intangibles. 

Training  Features 

Training  using  repetition.  Soldiers  were  asked  whether  the  various  training  events  use 
repetition  to  train.  The  three  highest  rated  events  for  using  repetition  in  training  were  gunnery 
(M=  4.32,  SD=. 91),  Combat  Life-Saver  Training/MSTC  training/trauma  lane  (M=4.09,  579=. 72), 
and  field  exercises  (M= 3.98,  SD= 1.1 1).  The  effective  use  of  this  feature  in  training  the 
intangibles  was  a  frequent  theme  (67%  of  sessions).  Some  Soldiers  suggested  that  repetition  is 
an  important  training  feature.  For  example,  one  Soldier  said  “Repetition  is  huge;  anything  you 
do  with  Soldiers  it  is  muscle  memory;  carrying  a  personal  weapon;  working  with  a  crew;  chair 
drills  in  an  office,  crew  drills  in  a  tank.”  Another  Soldier  said  “Repetition,  falls  back  to  get  a  guy 
repeating  the  motions  so  it  is  second  instinct.”  Other  Soldiers  believed  that  repetition  is 
somewhat  less  useful  in  training,  specifically  for  the  intangibles.  For  example,  one  Soldier  said 
“repetition  doesn’t  necessarily  get  at  it,  but  that’s  how  the  Anny  functions. . .  doesn’t  necessarily 
fit  the  intangibles.”  Similarly  another  Soldier  noted  that  repetition  can  be  helpful  for  establishing 
baseline  skills,  but  not  for  more  complex  application  of  those  skills  “. . .  repetition  gives  you  the 
how-you-do  piece  or  how  you  operate  as  a  section  -  but  doing  the  same  way  every  time  doesn’t 
build  a  team  that  can  react  to  any  situation.  But  first  they  must  leam  their  job.”  One  Soldier 
noted  that  the  problem  with  repetition  is  that  Soldiers  “know  what  to  expect.”  Another  Soldier 
distinguished  repetition  from  other  training  features: 

Repetition  to  train. .  .1  think  there  are  two  ways  people  learn  things,  rote  repetition 
or  a  significant  emotional  event.  The  preferred  method  is  for  repetition.  When 
we  have  enough  time  that  is  always  the  best.  When  you  throw  in  uncertainty  and 
stress  and  challenging  training  opportunities,  you  begin  to  approach  a  significant 
emotional  event. 

Thus,  repetition  appears  to  be  a  training  feature  that  supports  a  Soldier’s  ability  to  initially  react 
and  function  under  difficult  yet  commonly  experienced  circumstances;  a  coping  mechanism  of 
sorts.  At  the  same  time,  Soldiers  are  also  saying  that  repetition  may  work  against  the  natural 
inclination  to  adapt  or  innovate  when  confronted  with  challenging  or  complex  situations. 


27 


Incorporating  uncertainty  into  training.  Soldiers  were  asked  whether  the  various 
training  events  incorporate  uncertainty  into  training.  The  three  highest  rated  events  for 
incorporating  uncertainty  into  training  were  the  NTC  (M=4.62,  <SZ)=.89),  Ranger 
School/Mungadai  training  (M=4.50,  SD=. 65),  and  field  exercises  (M=4.09,  SD=.9 1 ).  The 
effective  use  of  this  feature  in  training  the  intangibles  was  a  frequent  theme  (72%  of  sessions). 
Soldiers  often  mentioned  the  training  of  uncertainty  while  describing  a  particular  training  event. 
For  example,  one  Soldier  said  “In  a  field  exercise  and  lanes,  that  is  where  you  need  ambiguity  - 
don’t  know  what  to  expect;  mix  it  up.”  Another  Soldier  described  the  effectiveness  of  the  three 
highest  rated  training  events: 

Uncertainty  -  NTC,  Mungadai  events  if  done  right,  (are)  the  top  tier,  then  below 
that  is  highest  level  field  exercise.  That  is  where  I  am  most  assessing  leadership 
in  the  organization  -  get  them  to  deal  with  uncertainty  and  ambiguity. 

Dealing  with  stress  in  training.  Soldiers  were  asked  whether  the  various  training  events 
incorporate  dealing  with  stress.  The  three  highest  rated  training  events  for  incorporating  dealing 
with  stress  were  Ranger  School/Mungadai  training  (M=4.67,  SD=. 49),  NTC  (M= 4.65,  SD=.74), 
and  field  exercises  (M=4.39,  SD=.6 9).  The  effective  use  of  this  feature  in  training  the 
intangibles  was  also  a  frequent  theme  (78%  of  sessions).  Soldiers  often  directly  mentioned  stress 
as  important  in  training  intangibles.  One  Soldier  said 

If  you  are  looking  at  trying  to  train  these,  you  want  to  incorporate  stress.  That  is 
where  you  will  be  able  to  see  if  he  has  the  will  or  grit  to  work  when  tired.  If  you 
can  incorporate  that,  you  will  find  the  intangibles. 

Soldiers  also  mentioned  various  training  events  where  stress  could  be  incorporated  into 
training.  One  Soldier  suggested  the  following  training  “More  rigorous  include  spur  rides  and 
Mungadai ’s  with  bigger  stressors  added;  then  complement  that  with  field  exercises  where  you 
really  stress  Soldiers  in  ambiguous  situations.”  Another  Soldier  commented  on  NTC  “That  30 
days  in  the  middle  is  often  times  worse  than  real  combat  as  far  as  living  conditions,  levels  of 
stress,  and  exhaustion.” 

One  Soldier  identified  the  importance  of  conducting  training  right: 

. . .  gunnery  can  be  stressful  if  you  do  it  right.  If  it’s  not  done  right  then  it’s 
not. .  .the  same  thing  with  an  field  training  exercise  (FTX).  An  FTX  can  be  great 
unless  it’s  done  improperly  then  its  garbage.  You  have  to  go  long  periods  of  time 
without  a  lot  of  sleep.  Starting  out  with  complex  tasks  and  making  them  simpler, 
or  else  what  you  get  is  a  blah  kind  of  training  and  Soldiers  know  it  and  have  no 
confidence  in  their  abilities. 

Difficult/challenging  training.  Soldiers  were  asked  whether  the  various  training  events 
are  difficult,  rigorous,  and  challenging  training  opportunities.  The  three  highest  rated  events  that 
encompass  difficult,  rigorous,  and  challenging  training  were  Ranger  School/Mungadai  training 
(M=4.80,  SD=A\),  NTC  (M= 4.59,  SD=. 78),  and  field  exercises  (M=4.20,  SD=. 85).  The 
effective  use  of  this  feature  in  training  the  intangibles  was  a  frequent  theme  (72%  of  sessions). 


28 


The  most  frequent  comments  concerning  difficult  training  concerned  pushing  people  to  and 
beyond  their  limits.  One  Soldier  identified  continuous,  lengthy  training  as  a  way  to  make 
training  difficult:  .  also  longer  term  -  FTX  is  a  week  but  NTC  is  like  four  -  it  pushes  people 

beyond  where  they  would  give  up  normally.”  Another  Soldier  said  “I  want  to  have  events  where 
I  can  just  test  you;  make  it  as  hard  as  possible.  That  is  where  you  test  will  -  the  longer  tenn 
grit.” 


Soldier  comments  that  described  difficult  training  also  spoke  about  approaching  or 
attaining  a  state  of  physical  exhaustion.  One  Soldier  said  “Mungadai  -  they  wear  your  a**  out. 
It’s  physically  challenging  and  pushes  you  as  a  group.  You  conduct  physically  demanding 
operations  over  a  period  of  time.”  Another  Soldier  put  it  simply  “Smoke  them  -  that’s  what  you 
have  to  do.”  Lastly,  Soldiers  mentioned  the  use  of  adverse  weather  or  training  conditions  in 
making  training  more  difficult.  For  example,  one  Soldier  said  “extended  over  24-48  hours  -  and 
with  weather  and  fatigue,  then  you  get  to  the  ‘grit’  part  of  it.” 

Negative  training  features.  Soldiers  were  asked  whether  the  various  training  events 
focus  too  much  on  qualifying.  Soldiers  responded  that  the  three  events  that  focus  too  much  on 
qualifying  are  gunnery  (M= 3.54,  SD=  1 .27),  Basic  Training  problem  solving  exercises  (M= 3.16, 
SZ)=1.05),  and  Combat  Life-Saver  Training/MSTC  training/trauma  lane  (M= 2.85,  SD=  1 .09). 
Also,  Soldiers  were  asked  whether  the  various  training  events  are  generally  a  check  the  box 
training  activity.  Soldiers  responded  that  the  three  events  known  to  be  a  check  the  box  training 
activity  are  PowerPoint  training  (M= 3.68,  SD=  1.29),  classroom  training  (M=3.58,  SD=  1.23), 
and  Resiliency  Training  (M=  3.35,  SZ)=1.07). 

Summary.  The  preceding  results  suggest  that  the  training  features  most  appropriate  for 
training  the  intangibles  incorporate  uncertainty,  stress,  and  making  training  difficult.  There  were 
mixed  views  on  the  utility  of  repetition  as  a  training  feature.  Soldiers’  comments  suggest  the 
training  events  that  rely  most  on  repetition  (e.g.,  gunnery)  may  be  more  appropriate  for  building 
baseline  skills  and  less  effective  at  training  intangibles.  The  three  training  events  possessing  the 
most  effective  training  features  were  Ranger  School/Mungadai  training,  the  NTC,  and  field 
exercises.  They  were  cited  specifically  by  interview  and  focus  group  Soldiers  as  being 
characterized  by  uncertainty  and  difficulty,  both  of  which  contribute  to  the  high  amount  of  stress 
experienced  by  some  individuals. 

Measuring  Intangibles  in  the  Field 

The  questionnaire  results  suggest  that  the  most  effective  training  events  for  the 
intangibles  are  Ranger  School,  the  NTC,  field  exercises,  OJT,  and  gunnery.  These  results  were 
consistent  with  Soldiers’  comments  throughout  the  interview  and  focus  group  sessions. 
Specifically,  Soldiers  were  asked  “Where  do  you  think  training  on  intangibles  could  fit  into 
existing  pre-deployment  training  while  adding  minimal  burden?”  Soldiers  tended  to  respond 
with  the  same  events  that  are  mentioned  above.  Another  frequent  theme  was  that  physical 
training  (PT)  was  effective  at  training  the  intangibles  (50%  of  sessions).  That  PT  is  effective  at 
training  intangibles  is  somewhat  at  odds  with  previous  findings  that  indicated  complex,  multi¬ 
task  field  (e.g.  Ranger  School,  NTC,  field  exercise)  situations  were  required  for  the  training  of 
intangibles.  Perhaps  reaching  and  passing  levels  of  physical  exhaustion  through  rigorous  PT 


29 


contributes  to  the  development  of  intangibles.  This  finding  opens  the  door  to  less  resource 
intensive  ways  of  attaining  Soldier  psychological  readiness  through  the  training  of  intangibles. 

Important  to  the  fielding  of  intangible  measures,  Soldiers  frequently  mentioned  that  there 
was  not  enough  time  for  training  (39%  of  sessions).  The  current  training  requirements  and 
deployment  cycle  does  not  support  a  recommendation  from  this  research  to  merely  add 
additional  training  on  the  intangibles  specifically.  Rather,  intangibles  training  should  be 
integrated  into  existing  training  requirements.  One  Soldier  said  “There  already  is  way  too  much 
in  pre-deployment.”  Another  Soldier  described  in  detail  the  problems  associated  with  current 
training  requirements  and  the  need  to  make  sure  that  basic  skills  training  is  done  prior  to  field 
exercise  events: 

When  it’s  time  to  go  to  that  FTX,  if  you  are  not  smart  along  the  way  in  doing 
training  along  the  way  at  every  opportunity  at  the  FTX  you  are  doing  more  of  the 
basic  training  to  get  you  to  a  certain  level.  As  opposed  to  here’s  the  training,  let’s 
start  throwing  in  the  mud  and  the  staying  up  for  30  hours  and  all  those  harsh 
environmental  conditions.  You  have  to  reach  a  certain  level  of  proficiency  before 
you  introduce  those  conditions. 

A  potential  implication  of  this  observation  is  that  Soldiers  ought  to  first  be  trained  on  basic 
tactical  and  technical  skills  under  nonnal  physical  environment  conditions.  It  is  then  appropriate 
(and  perhaps  more  effective)  to  change  physical  conditions  and  at  the  same  time  integrate 
training  on  intangibles  into  field  exercises. 

Measurement  Characteristics 

After  Action  Review.  In  focus  groups  and  interview  sessions,  Soldiers  were  asked 
“How  is  training  typically  evaluated?  After  training,  what  is  done  to  determine  whether  training 
was  a  success  and  that  Soldiers  were  trained  effectively?”  The  AARs  were  frequently  mentioned 
(at  least  once  in  67%  of  the  sessions).  Referring  to  the  NTC,  one  Soldier  described  the  type  of 
AAR  that  is  ineffective  “You  do  a  canned  3  ups  3  downs  AAR  which  isn’t  very  helpful.  But  in 
an  FTX  we  could  answer  the  ‘why’  more.”  Another  Soldier  mentioned  that  AARs  at  NTC  are 
good  because  they  are  constant,  “. . .  AAR  is  critical  and  one  of  the  things  about  NTC  is  you  do 
AARs  all  the  time.”  Another  Soldier  noted  the  value  of  field  exercise  AARs  “Field  exercise  is 
best  place  to  do  it  -  but  I’ve  seen  a  lot  of  them  done  poorly;  must  have  a  good  leader  driving  it.” 

Soldiers  noted  the  importance  of  having  a  good  leader  facilitating  AARs.  One  Soldier 
said  “AARs  are  useful  because  multiple  perspectives  come  out  -  facilitated  by  a  superior  -  it 
allows  a  unit  to  self  discover  strengths  and  weaknesses.  Superior  needs  to  ask  the  right  questions 
to  draw  out  the  discussion.”  Another  Soldier  said  that  leaders  need  to  “Facilitate,  but  make  them 
(Soldiers)  discuss  the  issues  amongst  themselves.  It  takes  an  experienced  leader  to  conduct  the 
AAR  process.”  An  implication  here  is  given  the  ‘intangible’  nature  of  intangibles;  an 
experienced  leader  is  most  likely  needed  to  draw  out  Soldier  observations  of  initiative,  will,  grit, 
and  hardiness  in  an  AAR  setting. 
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Rater  evaluation.  Soldiers  identified  multiple  different  types  of  raters  that  they  thought 
were  effective  at  observing  performance  on  the  intangibles  and  providing  feedback.  They  were 
also  asked  to  identify  the  accuracy  of  raters  and  their  time  available  for  observing  training. 
Composite  means  were  examined  to  specifically  examine  which  raters  were  most  accurate  or  had 
the  most  time  for  measuring  intangibles,  regardless  of  training  event.  To  perform  this  analysis, 
Soldier  scores  on  an  item  were  averaged  across  the  training  events.  Take,  for  example,  the  item 
“provides  an  environment  where  peers  can  accurately  evaluate  performance  on  intangibles.” 

This  item  had  10  different  scores  from  Soldiers  (i.e.,  one  for  each  of  the  different  training 
events).  These  10  scores  were  averaged  together  to  create  a  composite  mean  for  peers’  ability  to 
accurately  evaluate  performance  on  intangibles. 

Composite  means  indicated  that  peer  raters  (M= 3.47,  SD=. 5 1),  superior  raters  (M=  3.60, 
SD=.  58),  and  self  raters  (M= 3.54,  SD=. 62)  are  generally  perceived  to  be  about  the  same  in  their 
accuracy  at  observing  intangibles  during  training;  with  superiors  slightly  more  accurate.  Further, 
composite  means  indicated  that  peer  raters  (M=3.41,  SD=. 45),  superior  raters  (M=3.46, 

SD=.  59),  and  self  raters  (M=3.46,  SD=.6  0)  are  perceived  to  have  about  the  same  amount  of  time 
to  observe  intangibles  during  training. 

Soldiers  in  interviews  and  focus  groups  mentioned  several  rater  sources,  among  them 
superiors,  as  effective  at  measuring  intangibles.  And  the  most  frequently  mentioned  rater  source 
for  measuring  and  providing  feedback  on  the  intangibles  was  superiors.  Several  different  types 
of  superiors  were  mentioned  as  effective  raters,  such  as  the  first  line  supervisor,  unit  commander, 
platoon  leader,  squad  leader,  and  team  leaders.  During  interview  and  focus  group  sessions, 
Soldiers  mentioned  the  effectiveness  of  instructors/  observer  controllers  (44%  of  sessions)  and 
peer  evaluation  (28%  of  sessions).  Self-rating  and  subordinate  rating  sources  were  discussed, 
but  were  mentioned  infrequently.  One  point  of  commonality  among  the  rating  sources  deemed 
most  effective  at  rating  intangibles  is  that  they  have  had  or  are  currently  experiencing  the 
position  responsibilities  of  the  person  they  are  observing.  This  would  explain  why  superiors, 
instructors/ob server  controllers,  and  peers  (to  a  lesser  extent)  are  reported  to  be  effective 
observers  of  intangibles.  This  also  points  to  the  need  to  select  or  assign  experienced  and 
seasoned  personnel  for  the  responsibility  of  observing  and  assessing  intangibles  in  others. 

Training  Performance  Indicators  for  the  Intangibles 

Throughout  the  interviews  and  focus  groups,  two  general  themes  regarding  perfonnance 
indicators  emerged.  First,  Soldiers  mentioned  in  56%  of  sessions  that  the  intangibles  seemed  to 
be  value  or  trait-based.  This  is  consistent  with  the  literature  that  shows  that  intangibles  such  as 
hardiness  are  sometimes  referred  to  as  trait-based.  Second,  in  33%  of  the  sessions,  Soldiers 
mentioned  that  one  or  more  of  the  intangibles  seemed  related  to  another  one  of  the  intangibles. 
Again,  this  is  consistent  with  the  earlier  completed  literature  review.  Implications  of  the  first 
findings  may  be  that  it  will  be  difficult  for  training  on  intangibles  to  gain  unit  leader  acceptance 
(if  they  believe  an  intangible  is  not  subject  to  development).  As  to  the  second  finding,  it  may 
necessitate  combining  or  packaging  intangible  measures  so  that  the  unit  is  not  duplicating  its 
measurement  of  similar  constructs. 
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Initiative.  Soldiers  were  asked  to  identify  the  performance  indicators  for  initiative. 
There  were  five  themes  that  were  frequently  mentioned  as  perfonnance  indicators  for  initiative. 
An  additional  perfonnance  indicator  was  nearly  as  frequent  and  is  included  in  Table  3.  Table  3 
identifies  the  performance  indicators  that  were  mentioned,  the  percentage  of  sessions  that  the 
theme  was  mentioned  in,  and  a  representative  quote  for  each  theme. 

Table  3. 


Performance  Indicators  of  Initiative 


Performance  Indicators 

%of 

Sessions 

Representative  Quote 

Knows  responsibilities,  takes 
or  assumes 

responsibility/ownership 

50% 

In  the  absence  of  leadership  you  will  see  it  in  people  who 
take  the  leadership  position.  You  can’t  teach  someone 
how  to  take  initiative.  He  has  to  have  it  in  him  and  step 
up  in  absence  of  leadership  regardless  of  what  other 
people  think.  When  he  is  working  with  others,  they 
know  that  he  will  take  charge.  It  is  something  he  brings 
to  the  team. 

Anticipates  what  needs  to  be 
done 

44% 

Someone  with  initiative  can  make  an  educated  guess  on 
the  10-12  things  that  need  to  be  done,  regardless  of  the 
specifics.  See  that  a  lot  at  platoon  sergeant  (PSG)  and 
squad  leader  levels...  The  guys  able  to  visualize  the  next 
requirements,  they  just  do  better. 

Prepares  for  Next  Step 

28% 

A  person  who  thinks  about  what  is  next  and  not  just 
taking  care  of  current  problem  set.  And  that  is  hard  to 
do,  you  have  to  project.  If  you  can  anticipate  and  plan 
outside  the  current  threat,  to  counteract  a  potential  threat. 

Effectively  Uses  Time 

28% 

You  certainly  know  when  it  (initiative)  is  absent  -  very 
obvious.  If  they  lack  it,  they  will  chase  the  clock  all  the 
time. 

Leads  Peers 

28% 

When  you  come  to  your  company  -  I  see  an  NCO  sitting 
on  a  pallet  doing  nothing;  and  a  SPC  grabs  a  PFC  and  a  - 
10  manual  and  preventive  maintenance  checks  and 
services  (PMCS)  the  entire  vehicle  -  it  takes  an  hour  -  he 
wants  to  be  in  charge  and  wants  to  be  that  leader. 

Listens,  Understands  Big 
Picture/  Commander’s  intent 

22% 

The  person  understands  the  long  term  goals  of  the 
organization  and  understanding  of  what  little  things  he 
can  do  to  better  train  Soldiers.  Can  certainly  see  the  big 
picture  and  the  essence  of  what  we  are  trying  to  do  is  not 
inside  the  domed  tactical  operations  center  (TOC)  he  is 
in. 

Will.  Soldiers  were  asked  to  identify  the  perfonnance  indicators  for  will.  There  were 
three  themes  that  were  frequently  mentioned  as  performance  indicators  for  will.  Table  4 
identifies  the  performance  indicators  that  were  mentioned,  the  percentage  of  sessions  that  the 
theme  was  mentioned  in,  and  a  representative  quote  for  each  theme. 
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Table  4. 


Performance  Indicators  of  Will 

%of 

Performance  Indicators  Sessions 
Persistence/How  a  67% 

Soldier  reacts  to 
adversity 


_ Representative  Quote _ 

(They)  keep  going  and  not  just  throw  their 
hands  up  and  say  this  is  a  problem  I’ve  never 
encountered;  I’m  not  sure  what  to  do.  They  just 
keep  trying  to  tackle  the  problem.  When  they 
run  into  another  problem  they  keep  at  it;  they 
stay  up.  I’m  going  against. .  .obviously  we  need 
to  train  Soldiers  to  have  a  sleep  plan  and  take 
care  of  themselves,  but  it’s  that  guy  who  refuses 
to  go  to  sleep  until  it’s  done  or  misses  a  meal 
and  refuses  to  give  up;  doesn’t  take  a  knee  or 
the  easy  way  out,  and  continues  to  try  to  make 
himself  better. 

When  time  gets  hard,  they  are  able  to  keep 
going.  I  have  seen  individuals  who  have  the 
skill  to  do  it,  but  choose  not  to. 


Lack  of  Complaint/ 
Can-Do  Attitude 


Demonstrating  a  Will 
to  Win 


22%  It’s  the  guy  that  doesn’t  complain  -  whatever 
you  throw  at  him  he  just  does  it. 

The  one  that  doesn’t  have  will  is  the  one  that,  at 
mission  time,  will  not  want  to  go,  he’ll  be 
hiding  out  in  his  room.  On  a  mission  he  is  the 
one  lagging  behind,  wanting  to  stay  with  the 
trucks  rather  than  dismount.  The  one  with  will 
is  ready  to  go,  standing  up,  and  taking  point. 

22%  To  me,  I  see  will  as  the  competitive  spirit.  You 
can  kind  of  teach  that  into  them;  to  set  the  habit 
to  want  to  win  or  to  succeed.  There  are  some 
unhealthy  and  healthy  levels  of  competitive 
spirit  -  at  unhealthy  levels  they  are  cheating  to 
win  -  winning  at  all  costs.  In  reality,  it  is  how 
we  win  as  well.  You  can’t  always  pick  that  out 
watching  them  walk  down  the  hallway. 


Grit.  Soldiers  were  asked  to  identify  the  perfonnance  indicators  for  grit.  There  were 
two  themes  that  were  mentioned  as  perfonnance  indicators  for  grit.  Table  5  identifies  the 
performance  indicators  that  were  mentioned,  the  percentage  of  sessions  that  the  theme  was 
mentioned  in,  and  a  representative  quote  for  each  theme. 
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Table  5. 


Performance  Indicators  of  Grit 


%of 

Perfonnance  Indicators 

Sessions 

Representative  Quote 

Persistence 

44% 

If  they  have  failure  and  continue 
to  work  hard  -  the  Army  is  a 
great  social  Petri  dish  for  this. 

See  a  kid  who  gets  in  trouble  - 
can  work  harder  to  improve 
himself. . .  You  want  them  to  see 
the  mistake,  improve 
themselves. 

Dedication  to  the  Anny 

28% 

We  always  ask  -  is  the 
organization  better  from  the  time 
you  came  in  to  the  time  you 
leave?  A  person  may  think  this 
for  themselves,  but  more 
honestly  they  should  ask  their 
peers  for  their  perspective.  So 
identifying  someone  who  is 
motivated  to  improve  the 
organization  -  relates  to  grit. 

Hardiness.  Soldiers  were  asked  to  identify  the  perfonnance  indicators  for  hardiness. 
Persistence  was  the  only  frequent  theme  (39%  of  sessions)  identified  as  a  hardiness  perfonnance 
indicator.  One  Soldier  said  “If  you  train  and  train  and  actually  deploy  -  all  you  trained  for  is 
working  well,  and  you  have  one  thing  that  goes  wrong  and  kinks  the  whole  system  -  if  you  can’t 
recover  from  that,  it  kills  the  entire  deployment  or  rotation  or  whatever  you  are  doing.  It  goes 
back  to  being  an  individual  -  if  you  can’t  pick  yourself  up,  your  buddy  needs  to  pick  you  up.” 
Another  Soldier  described  how  he  thought  hardiness  could  be  measured  “Hardiness  is  easily 
testable  -  we  put  people  in  so  many  jobs  they  don’t  want  to  do.  Or  go  do  a  task  that  is 
completely  beneath  you,  or  that  you  hate.  Do  you  have  the  attitude  to  see  there  is  something 
good  that  will  come  out  of  it  -  something  to  learn  or  someone  to  learn  from.  That  positive,  but 
realistic  attitude  could  be  measured.” 


Conclusions 

Ranger  School,  NTC,  and  field  exercises  were  rated  as  the  most  effective  training  events 
for  measuring  initiative,  will,  grit,  and  hardiness.  Soldiers  also  rated  these  training  events  most 
effective  in  both  training  methods  and  training  features.  Focus  groups  and  interviews  also 
identified  them  as  training  events  that  create  the  conditions  for  the  testing  of  an  individual 
Soldier’s  initiative,  will,  grit,  and  hardiness.  Considerations  for  employing  and  integrating  the 
training  of  intangibles  into  these  events  are  discussed  below. 
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Ranger  School  is  a  training  program  for  which  mostly  combat  anns  personnel  are 
selected  to  attend.  It  is  not  a  unit  home  station  training  opportunity  that  could  be  integrated 
directly  into  BCT  training.  Yet  the  training  conditions  and  experiences  of  Ranger  school  (e.g., 
difficult/rigorous  training)  provide  indicators  of  what  BCT  trainers  can  do  to  enhance  Soldier 
psychological  readiness.  Commanders,  too,  may  want  to  request  Ranger  school  slots  and 
program  a  number  of  individuals  to  attend  it  early  in  the  unit’s  ARFORGEN  training  cycle.  And 
it  might  be  given  to  an  installation  level  mini  Ranger  school,  designed  to  emulate  some  of  the 
characteristics  of  Ranger  school.  Units  can  also  elect  to  plan  and  execute  Mungadai  training  at 
home  station.  Field  exercise  and  NTC  training  events  are  also  effective  events  for  instilling  and 
measuring  the  intangibles.  For  field  exercises,  commanders  and  training  management  personnel 
would  need  to  integrate  the  training  features  and  methods  of  training  previously  reported  so  that 
Soldier  initiative,  will,  grit,  and  hardiness  can  be  experienced,  tested,  and  measured.  Most  likely 
superiors  and/or  designated  observers  would  need  to  be  familiarized  with  tools  for  measuring  the 
intangibles  (a  forthcoming  product  of  this  research).  As  the  unit  commander  and  staff  typically 
run  and  control  field  exercises,  they  would  be  in  a  position  to  ensure  that  the  training  conditions 
necessary  for  the  testing  and  development  of  intangibles  are,  in  fact,  created.  A  focal  point  of 
AARs  could  also  be  individual  and  collective  (e.g.,  unit)  assessments  of  initiative,  will,  grit,  and 
hardiness.  The  challenge  of  integrating  intangibles  into  field  exercises  is  gaining  commander, 
staff,  and  unit  attention  to  it  and  adapting  training  management  processes  to  incorporate  it  into 
existing  unit  training. 

The  NTC  training  events  share  many  of  the  same  characteristics  of  field  exercises  yet 
much  of  the  training  is  resourced  and  supported  by  the  NTC  itself.  Due  to  the  well-resourced 
and  extended  timeframe  of  NTC  rotations,  it  already  encompasses  the  conditions  by  which  the 
strength  of  a  Soldier’s  will,  grit,  and  hardiness  together  with  their  ability  to  exercise  initiative 
will  be  tested.  What  remains  is  for  the  unit  commander  to  communicate  to  the  NTC  operations 
group  that  s/he  would  like  their  units  to  receive  feedback  on  the  intangibles  during  their  NTC 
rotation.  The  commander  could  provide  the  operations  group  with  field  ready  measures  (a 
forthcoming  product  of  this  research)  and  request  that  observer/trainers  discuss  the  measurement 
of  intangibles  with  their  counterpart  unit  leader.  Thus,  the  commander  could  integrate  training 
on  intangibles  within  the  NTC  rotation. 

That  PT  was  mentioned  as  a  way  of  training  intangibles  should  not  be  overlooked.  It  is  a 
low  resource-intensive  means  of  creating  the  conditions  for  intangibles  to  be  in  evidence. 
Rucksack  marches,  demanding  obstacle  courses,  and  even  extended  unit  runs  may  evoke 
opportunities  for  a  Soldier’s  will,  for  example,  to  be  tested. 

Supervisors  and  unit  commanders  were  often  identified  as  best  qualified  to  measure  the 
intangibles  as  well  as  facilitate  AARs  that  integrate  Soldier  and  unit  understanding  of  intangibles 
and  how  they  had  an  impact  on  mission  accomplishment.  This  places  a  burden  on  the  more 
experienced  leaders  of  a  unit.  At  the  same  time,  leaders  at  all  echelons,  from  team  leader  to 
commander,  in  positions  of  direct  Soldier  observation  were  deemed  suitable  for  the  measurement 
of  intangibles.  Thus,  the  additional  time  needed  to  measure  intangibles  can  be  distributed  across 
the  unit’s  leadership.  So,  too,  dedicated  observer/trainers  are  a  resource  commanders  might  plan 
for  more  often  as  part  of  field  exercises,  for  the  expressed  purpose  of  supporting  intangible 
measurement  and  feedback. 
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Lastly,  BCT  identification  of  intangible  perfonnance  indicators  that  are  mostly  consistent 
with  the  literature  review  indicates  that  units  recognize  the  importance  of  the  intangibles  selected 
for  measurement  by  this  research.  Thus,  the  combination  of  performance  indicators  and 
representative  quotes,  with  measures  from  the  literature  review,  provides  a  sound  foundation  for 
the  development  of  intangible  measurement  tools  readily  applied  in  a  BCT  training  environment. 
This  is  the  next  step  of  the  research. 
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APPENDIX  A 

PHASE  II  QUESTIONNAIRE 
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Start  each  sentence  with  the  following  different  Training  Events 
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Please  read  through  the  statements  by  starting  each 
sentence  with  one  of  the  training  events  to  the  right, 
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and  then  reading  the  sentences  located  below.  Rate 
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your  agreement  with  each  statement  by  responding 
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with  one  of  the  following  response  options  (0=N/A  or 

No  Response,  1=  Strongly  disagree,  2=  Disagree,  3= 
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Neutral,  4=Agree,  5=  Strongly  Agree 
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Training  Methods 

provides  practice  and  experience  in  training 

provides  realism  during  training 

provides  multiple,  different  opportunities  to  apply 
knowledge  and  skills 

offers  a  collaborative  or  Team-based  training 
experience 

allows  Soldiers  to  explain  their  grasp  of  the  training 
exercise  and  discuss  examples  of  their  performance 
during  training 

provides  a  support  structure  (e.g.,  peers  or  a  leader) 
helpful  in  building  knowledge  of  simple  tasks  into 
complex  skills 

has  an  instructor  or  leader  who  is  able  to  guide  and 
challenge  development  during  training 

involves  understanding,  problem  solving,  and  making 
sense  of  new  events  rather  than  the  memorization  of 
facts 

allows  participants  to  critique  their  performance  and 
assess  their  ability  to  learn  during  training 

Training  Features 

uses  repetition  to  train 

incorporates  uncertainty  into  training 

incorporates  dealing  with  stress 

is  a  difficult,  rigorous,  and  challenging  training 
opportunity 

focuses  too  much  on  qualifying 

is  generally  a  check  the  box  training  activity 
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Start  each  sentence  with  the  following  different  Training  Environments 


Please  read  through  the  statements  by  starting  each  sentence 
with  one  of  the  training  events  to  the  right  and  then  reading  the 
sentences  located  below.  Rate  your  agreement  with  each 
statement  by  responding  with  one  of  the  following  response 
options  (0=N/A  or  No  Response,  1=  Strongly  disagree,  2= 
Disagree,  3=  Neutral,  4=Agree,  5=  Strongly  Agree 
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Rater  and  Time  Concerns 

provides  an  environment  where  peers  can  accurately  evaluate 
performance  on  intangibles 

provides  a  sufficient  amount  of  time  for  peers  to  observe  and 
critique  performance  on  intangibles 

provides  an  environment  where  superiors  can  accurately 
evaluate  performance  on  intangibles 

provides  a  sufficient  amount  of  time  for  superiors  to  observe  and 
critique  performance  on  intangibles 

provides  an  environment  where  Soldiers  can  accurately  evaluate 
their  own  performance  on  intangibles 

provides  a  sufficient  amount  of  time  for  Soldiers  to  observe  and 
critique  their  own  performance  on  intangibles 

provides  a  sufficient  amount  of  time  for  superiors/instructors  to 
give  individual  feedback 

Intangibles 

is  an  excellent  environment  to  measure  Soldier's  Initiative 

is  an  excellent  environment  to  measure  Soldier's  Will 

is  an  excellent  environment  to  measure  Soldier's  Grit 

is  an  excellent  environment  to  measure  Soldier's  Hardiness 
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APPENDIX  B 

INTERVIEW  AND  FOCUS  GROUP  PROTOCOL 
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Preparatory  Skill  Set  for  Brigade  Combat  Teams  (BCT)  Protocol 


Session  Information 


Date: _ 

Time: _ 

Interviewer: _ 

Interviewee’s  Title/Position: _ 

Introduction  and  Research  Purpose 


Good  morning/good  afternoon  and  thank  you  for  taking  the  time  to  participate  in  this  focus  group/interview.  My  name  is 

_ with  ICF  International.  I  am  part  of  a  research  team  that  has  been  contracted  by  the  U.S.  Anny  Research 

Institute  (ARI)  to  identify  important  skill  sets  to  Soldier  mission  readiness.  The  skill  set  of  interest  in  this  research  is  collectively 
tenned  intangibles;  specifically  the  intangibles  Initiative,  will,  grit,  and  hardiness.  Provide  definitions  of  intangibles. 

•  Initiative  -  the  willingness  to  act  in  the  absence  of  orders,  when  existing  orders  no  longer  fit  the  situation,  or  when  unforeseen 
opportunities  or  threats  arise. 

•  Will  -  inner  drive  that  compels  Soldiers  to  keep  going,  even  when  exhausted,  hungry,  afraid,  cold,  and  wet. 

•  Grit  -  perseverance  and  passion  for  long-tenn  goals.  Grit  entails  working  strenuously  in  the  face  of  challenges  and  maintaining 
effort  and  interest  over  years  despite  failure,  adversity,  and  plateaus  in  progress. 

•  Hardiness  -  a  specific  set  of  attitudes  and  skills  that  provide  the  courage,  motivation,  and  strategies  leading  to  resilience  and 
growth  in  stressful  circumstances. 

Further,  we  will  be  asking  questions  concerning  the  effectiveness  of  various  training  methods  and  training  features.  The  infonnation 
you  provide  today  will  be  applied  to  better  develop,  execute,  and  assess  the  training  of  intangibles. 

The  interview  session  (focus  group)  will  take  60  minutes  (90  minutes)  to  complete. 
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Privacy  Act  Statement  &  Consent  Form 


Please  note  that  your  participation  is  voluntary  -  there  are  no  consequences  if  you  choose  not  to  participate.  Everything  you  say  will 
remain  confidential.  We  will  be  transcribing  your  responses  with  laptops  or  digitally  recording  your  responses  with  a  voice  recorder, 
but  our  analysis  and  reporting  of  your  responses  will  be  at  the  group  or  aggregate  level — not  at  the  individual  level.  No  information 
collected  or  response  will  be  attributed  or  linked  to  any  one  individual. 

To  more  fully  explain  the  confidentiality  process  and  how  we  will  be  using  the  information  you  provide  today,  I  have  a  privacy 
statement  and  consent  form  for  you  to  read  over.  Please  take  a  few  minutes  to  read  over  both  documents.  If  you  choose  to  participate, 
please  sign  the  second  page  of  the  consent  form  and  indicate  that  you  are  over  18  years  old  and  are  voluntarily  agreeing  to  participate. 
Please  let  me  know  if  you  have  any  questions  about  the  privacy  statement,  consent  form,  or  the  session  today.  (Wait  until  it  looks  as 
though  everyone  has  finished  reading  and  then  ask  for  the  signed  consent  forms). 

Do  you  have  any  questions  for  me  at  this  time  either  in  terms  of  the  content  of  our  conversation  or  anything  else?  (. Answer  any 
questions  that  may  arise).  We  have  a  questionnaire  for  you  to  fill  out  ( hand  out  questionnaire).  Please  read  through  the  instructions 
and  take  a  few  minutes  to  fill  out  the  questionnaire.  After  you  are  finished  with  the  questionnaire,  we  will  begin  our  discussion. 

In  this  section,  we  will  discuss  the  training  methods  and  learning  theories  used  in  existing  exercises,  venues,  and  activities, 

o  Of  the  training  methods  listed  on  the  questionnaire,  which  ones  do  you  feel  are  most  essential  in  effectively  training 
intangibles  such  as  initiative,  will,  grit,  and  hardiness? 

■  Why  are  the  ones  you  identified  more  important  than  the  others? 

o  Of  the  training  features  listed  on  the  questionnaire,  which  ones  do  you  feel  are  most  essential  in  effectively  training 
intangibles  such  as  initiative,  will,  grit,  and  hardiness? 

■  Why  are  the  ones  you  identified  more  important  than  the  others? 

o  Think  of  the  most  effective  training  event  you’ve  ever  participated  in.  Why  was  this  so  effective  for  you? 

■  Did  it  utilize  any  of  the  training  methods  or  features? 
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In  this  second  section,  we  will  discuss  what  measurement  characteristics  are  most  appropriate  and  how  measurement  of 
intangibles  can  be  accomplished  and  adapted  to  the  field. 

o  How  is  training  typically  evaluated?  After  training,  what  is  done  to  determine  whether  training  was  a  success  and  that 
Soldiers  were  trained  effectively? 

o  What  training  events  usually  incorporate  individual  feedback  from  superiors? 

■  How  much  time  do  superiors  spend  providing  individual  feedback  on  training  performance? 

■  What  type  of  feedback  do  superiors  provide  following  training? 

o  Who  is  most  accurate  in  observing  and  providing  ratings  during  training  (e.g.,  self,  peers,  superiors,  and  independent 
instructors)? 

■  Does  the  most  accurate  group  also  have  time  available  to  provide  ratings  and  feedback? 

o  Where  do  you  think  training  on  initiative  could  fit  into  existing  pre-deployment  training  while  adding  minimal  burden? 

■  In  which  of  the  training  events  provided  would  be  the  best  setting  for  training  initiative? 

o  Where  do  you  think  training  on  will  could  fit  into  existing  pre-deployment  training  while  adding  minimal  burden? 

■  In  which  of  the  training  events  provided  would  be  the  best  setting  for  training  will? 

o  Where  do  you  think  training  on  grit  could  fit  into  existing  pre-deployment  training  while  adding  minimal  burden? 

■  In  which  of  the  training  events  provided  would  be  the  best  setting  for  training  grit? 

o  Where  do  you  think  training  on  hardiness  could  fit  into  existing  pre-deployment  training  while  adding  minimal  burden? 

■  In  which  of  the  training  events  provided  would  be  the  best  setting  for  training  hardiness? 
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In  this  final  section,  we  will  discuss  the  training  performance  indicators  for  the  selected  intangibles. 

o  During  training,  what  are  the  indicators  that  distinguish  someone  who  has  initiative  from  someone  who  doesn’t? 

o  During  training,  what  are  the  indicators  that  distinguish  someone  who  has  will  from  someone  who  doesn’t? 

o  During  training,  what  are  the  indicators  that  distinguish  someone  who  has  grit  from  someone  who  doesn’t? 

o  During  training,  what  are  the  indicators  that  distinguish  someone  who  has  hardiness  from  someone  who  doesn’t? 

That  concludes  the  questions  that  we  had  prepared  for  you. 

De-Briefing: 

Thank  you  again  for  your  time  and  participation.  Your  comments  have  been  very  helpful.  Our  data-collection  is  on-going.  There  will 
be  a  report  issued  at  the  conclusion  of  this  project.  If  you  are  interested  in  receiving  a  copy,  please  provide  us  with  your  e-mail 
address  and  we  will  send  you  one  once  it  is  complete. 
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APPENDIX  C 

QUANTITATIVE  RESULTS  -  TRAINING  METHODS 
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Statement 


Training  Event 

_ Quantitative  Results  -  Training  Methods _ 

NTC  provides  practice  and  experience  in  training. 

A  Field  exercise  provides  practice  and  experience  in  training. 

On-the-job  training  provides  practice  and  experience  in  training. 

Ranger  School/Mungadai  training  provides  practice  and  experience  in  training. 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  provides  practice 
and  experience  in  training. 

Gunnery  provides  practice  and  experience  in  training. 

Resiliency  training  provides  practice  and  experience  in  training. 

Basic  training  problem  solving  exercises  provides  practice  and  experience  in 
training. 

Classroom  training  provides  practice  and  experience  in  training. 

PowerPoint  training  provides  practice  and  experience  in  training. 
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Statement 


Training  Event 

Quantitative  Results  -  Training  Methods 

Count 

Range 

Mean 

Standard 

Deviation 

NTC  provides  realism  during  training. 

46 

4 

4.48 

0.78 

Ranger  School/Mungadai  training  provides  realism  during  training. 

46 

2 

4.20 

0.86 

A  Field  exercise  provides  realism  during  training. 

46 

3 

4.11 

0.69 

c 

QJ 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  provides  realism 
during  training. 

46 

3 

4.07 

0.82 

S 

QJ 

CS 

On-the-job  training  provides  realism  during  training. 

46 

4 

4.07 

0.89 

5? 

Gunnery  provides  realism  during  training. 

46 

3 

3.63 

0.92 

Basic  training  problem  solving  exercises  provides  realism  during  training. 

3.24 

0.79 

Resiliency  training  provides  realism  during  training. 

46 

3 

2.91 

0.88 

Classroom  training  provides  realism  during  training. 

46 

3 

2.38 

0.98 

PowerPoint  training  provides  realism  during  training. 

2.09 

0.94 

C-3 


Statement 


Training  Event 

_ Quantitative  Results  -  Training  Methods _ 

NTC  provides  multiple,  different  opportunities  to  apply  knowledge  and  skills. 

A  Field  exercise  provides  multiple,  different  opportunities  to  apply  knowledge 
and  skills. 

Ranger  School/Mungadai  training  provides  multiple,  different  opportunities  to 
apply  knowledge  and  skills. 

On-the-job  training  provides  multiple,  different  opportunities  to  apply 
knowledge  and  skills. 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  provides  multiple, 
different  opportunities  to  apply  knowledge  and  skills. 

Gunnery  provides  multiple,  different  opportunities  to  apply  knowledge  and 
skills. 

Basic  training  problem  solving  exercises  provides  multiple,  different 
opportunities  to  apply  knowledge  and  skills. 

Resiliency  training  provides  multiple,  different  opportunities  to  apply 
knowledge  and  skills. 

Classroom  training  provides  multiple,  different  opportunities  to  apply 
knowledge  and  skills. 

PowerPoint  training  provides  multiple,  different  opportunities  to  apply 
knowledge  and  skills. 


Count 

Range 

Mean 

Standard 

Deviation 

4 

4.52 

0.84 

46 

3 

4.38 

0.81 

46 

2 

4.33 

0.72 

46 

3 

4.15 

0.76 

46 

4 

3.95 

0.91 

46 

4 

3.85 

1.00 

46 

4 

3.53 

0.86 

46 

4 

3.12 

0.88 

3 

2.96 

0.89 

46 

3 

2.34 

0.91 

C-4 


Statement 


Training  Event 

Quantitative  Results  -  Training  Methods 

Count 

Range 

Mean 

Standard 

Deviation 

NTC  offers  a  collaborative  or  team-based  training  experience. 

4.63 

0.74 

A  Field  exercise  offers  a  collaborative  or  team-based  training  experience. 

46 

2 

4.38 

0.58 

Ranger  School/Mungadai  training  offers  a  collaborative  or  team-based  training 
experience. 

46 

3 

4.27 

1.10 

Gunnery  offers  a  collaborative  or  team-based  training  experience. 

4.03 

0.86 

On-the-job  training  offers  a  collaborative  or  team-based  training  experience. 

3 

3.82 

0.78 

Basic  training  problem  solving  exercises  offers  a  collaborative  or  team-based 
training  experience. 

46 

4 

3.63 

1.06 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  offers  a 
collaborative  or  team-based  training  experience. 

46 

4 

3.58 

1.10 

Resiliency  training  offers  a  collaborative  or  team-based  training  experience. 

46 

3 

3.03 

0.82 

Classroom  training  offers  a  collaborative  or  team-based  training  experience. 

2.76 

1.07 

PowerPoint  training  offers  a  collaborative  or  team-based  training  experience. 

3 

2.35 

0.87 
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Statement 


Training  Event 

_ Quantitative  Results  -  Training  Methods _ 

NTC  allows  Soldiers  to  explain  their  grasp  of  the  training  exercise  and  discuss 
examples  of  their  performance  during  training. 

A  Field  exercise  allows  Soldiers  to  explain  their  grasp  of  the  training  exercise 
and  discuss  examples  of  their  perfonnance  during  training. 

On-the-job  training  allows  Soldiers  to  explain  their  grasp  of  the  training 
exercise  and  discuss  examples  of  their  performance  during  training. 

Gunnery  allows  Soldiers  to  explain  their  grasp  of  the  training  exercise  and 
discuss  examples  of  their  perfonnance  during  training. 

Ranger  School/Mungadai  training  allows  Soldiers  to  explain  their  grasp  of  the 
training  exercise  and  discuss  examples  of  their  performance  during  training. 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  allows  Soldiers  to 
explain  their  grasp  of  the  training  exercise  and  discuss  examples  of  their 
performance  during  training. 

Basic  training  problem  solving  exercises  allows  Soldiers  to  explain  their  grasp 
of  the  training  exercise  and  discuss  examples  of  their  perfonnance  during 
training. 

Classroom  training  allows  Soldiers  to  explain  their  grasp  of  the  training 
exercise  and  discuss  examples  of  their  performance  during  training. 

Resiliency  training  allows  Soldiers  to  explain  their  grasp  of  the  training 
exercise  and  discuss  examples  of  their  perfonnance  during  training. 

PowerPoint  training  allows  Soldiers  to  explain  their  grasp  of  the  training 
exercise  and  discuss  examples  of  their  performance  during  training. 


Count 

Range 

Mean 

Standard 

Deviation 

4.42 

0.87 

4.11 

0.81 

46 

2 

3.93 

0.65 

3 

3.93 

0.76 

3 

3.73 

1.03 

3 

3.70 

0.89 

46 

2 

3.64 

0.65 

3.54 

0.89 

3.15 

0.94 

3 

2.88 

0.93 
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Statement 


Training  Event 

_ Quantitative  Results  -  Training  Methods _ 

NTC  provides  a  support  structure  (e.g.,  peers  or  a  leader)  helpful  in  building 
knowledge  of  simple  tasks  into  complex  skills. 

A  Field  exercise  provides  a  support  structure  (e.g.,  peers  or  a  leader)  helpful  in 
building  knowledge  of  simple  tasks  into  complex  skills. 

Ranger  School/Mungadai  training  provides  a  support  structure  (e.g.,  peers  or  a 
leader)  helpful  in  building  knowledge  of  simple  tasks  into  complex  skills. 

On-the-job  training  provides  a  support  structure  (e.g.,  peers  or  a  leader)  helpful 
in  building  knowledge  of  simple  tasks  into  complex  skills. 

Gunnery  provides  a  support  structure  (e.g.,  peers  or  a  leader)  helpful  in  building 
knowledge  of  simple  tasks  into  complex  skills. 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  provides  a  support 
structure  (e.g.,  peers  or  a  leader)  helpful  in  building  knowledge  of  simple  tasks 
into  complex  skills. 

Basic  training  problem  solving  exercises  provides  a  support  structure  (e.g., 
peers  or  a  leader)  helpful  in  building  knowledge  of  simple  tasks  into  complex 
skills. 

Resiliency  training  provides  a  support  structure  (e.g.,  peers  or  a  leader)  helpful 
in  building  knowledge  of  simple  tasks  into  complex  skills. 

Classroom  training  provides  a  support  structure  (e.g.,  peers  or  a  leader)  helpful 
in  building  knowledge  of  simple  tasks  into  complex  skills. 

PowerPoint  training  provides  a  support  structure  (e.g.,  peers  or  a  leader)  helpful 
in  building  knowledge  of  simple  tasks  into  complex  skills. 


Count 

Range 

Mean 

Standard 

Deviation 

4.30 

0.98 

4.25 

0.81 

4.07 

0.96 

4.02 

0.69 

3 

3.90 

0.74 

3 

3.82 

0.72 

46 

4 

3.46 

0.89 

3.27 

0.91 

3.20 

0.95 

2.70 

1.04 
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Statement 


Training  Event 

_ Quantitative  Results  -  Training  Methods _ 

Ranger  School/Mungadai  training  has  an  instructor  or  leader  who  is  able  to 
guide  and  challenge  development  during  training. 

NTC  has  an  instructor  or  leader  who  is  able  to  guide  and  challenge 
development  during  training. 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  has  an  instructor  or 
leader  who  is  able  to  guide  and  challenge  development  during  training. 

A  Field  exercise  has  an  instructor  or  leader  who  is  able  to  guide  and  challenge 
development  during  training. 

Gunnery  has  an  instructor  or  leader  who  is  able  to  guide  and  challenge 
development  during  training. 

Basic  training  problem  solving  exercises  has  an  instructor  or  leader  who  is  able 
to  guide  and  challenge  development  during  training. 

On-the-job  training  has  an  instructor  or  leader  who  is  able  to  guide  and 
challenge  development  during  training. 

Classroom  training  has  an  instructor  or  leader  who  is  able  to  guide  and 
challenge  development  during  training. 

Resiliency  training  has  an  instructor  or  leader  who  is  able  to  guide  and 
challenge  development  during  training. 

PowerPoint  training  has  an  instructor  or  leader  who  is  able  to  guide  and 
challenge  development  during  training. 


Count 

Range 

Mean 

Standard 

Deviation 

46 

3 

4.36 

0.93 

46 

4 

4.27 

0.96 

46 

3 

4.26 

0.79 

46 

3 

4.18 

0.84 

3 

3.95 

0.85 

46 

3 

3.94 

0.79 

3 

3.89 

0.86 

46 

4 

3.83 

0.85 

46 

4 

3.50 

1.02 

46 

4 

3.09 

1.12 
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Statement 


Training  Event 

Quantitative  Results  -  Training  Methods 

Count 

Range 

Mean 

Standard 

Deviation 

NTC  involves  understanding,  problem  solving,  and  making  sense  of  new 
events  rather  than  the  memorization  of  facts. 

46 

4 

4.51 

0.92 

A  Field  exercise  involves  understanding,  problem  solving,  and  making  sense  of 
new  events  rather  than  the  memorization  of  facts. 

46 

3 

4.36 

0.81 

Ranger  School/Mungadai  training  involves  understanding,  problem  solving, 
and  making  sense  of  new  events  rather  than  the  memorization  of  facts. 

46 

3 

4.20 

0.94 

On-the-job  training  involves  understanding,  problem  solving,  and  making 
sense  of  new  events  rather  than  the  memorization  of  facts. 

3 

3.89 

0.82 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  involves 
understanding,  problem  solving,  and  making  sense  of  new  events  rather  than 
the  memorization  of  facts. 

46 

4 

3.79 

0.80 

Basic  training  problem  solving  exercises  involves  understanding,  problem 
solving,  and  making  sense  of  new  events  rather  than  the  memorization  of  facts. 

3 

3.79 

0.82 

Gunnery  involves  understanding,  problem  solving,  and  making  sense  of  new 
events  rather  than  the  memorization  of  facts. 

46 

4 

3.35 

1.17 

Resiliency  training  involves  understanding,  problem  solving,  and  making  sense 
of  new  events  rather  than  the  memorization  of  facts. 

46 

4 

3.33 

0.92 

Classroom  training  involves  understanding,  problem  solving,  and  making  sense 
of  new  events  rather  than  the  memorization  of  facts. 

46 

4 

3.11 

1.05 

Power  Point  Training  involves  understanding,  problem  solving,  and  making 
sense  of  new  events  rather  than  the  memorization  of  facts. 

46 

3 

2.48 

1.07 
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Statement 


Training  Event 

_ Quantitative  Results  -  Training  Methods _ 

NTC  allows  Soldiers  to  critique  their  performance  and  assess  their  ability  to 
learn  during  training. 

A  Field  exercise  allows  Soldiers  to  critique  their  perfonnance  and  assess  their 
ability  to  leam  during  training. 

Gunnery  allows  Soldiers  to  critique  their  performance  and  assess  their  ability  to 
learn  during  training. 

Ranger  School/Mungadai  training  allows  Soldiers  to  critique  their  perfonnance 
and  assess  their  ability  to  learn  during  training. 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  allows  Soldiers  to 
critique  their  perfonnance  and  assess  their  ability  to  learn  during  training. 

On-the-job  training  allows  Soldiers  to  critique  their  performance  and  assess 
their  ability  to  leam  during  training. 

Basic  training  problem  solving  exercises  allows  Soldiers  to  critique  their 
performance  and  assess  their  ability  to  leam  during  training. 

Classroom  training  allows  Soldiers  to  critique  their  performance  and  assess 
their  ability  to  leam  during  training. 

Resiliency  training  allows  Soldiers  to  critique  their  perfonnance  and  assess 
their  ability  to  leam  during  training. 

PowerPoint  training  allows  Soldiers  to  critique  their  performance  and  assess 
their  ability  to  leam  during  training. 


Count 

Range 

Mean 

Standard 

Deviation 

46 

4 

4.43 

0.91 

46 

3 

4.34 

0.83 

46 

3 

4.20 

0.76 

46 

3 

4.13 

0.99 

46 

2 

3.98 

0.71 

3 

3.89 

0.80 

46 

3 

3.48 

0.91 

46 

4 

3.18 

1.02 

46 

4 

3.15 

0.97 

46 

4 

2.63 

1.09 
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QUANTITATIVE  RESULTS  -  TRAINING  FEATURES 


D-l 


Training  Event 

Quantitative  Results  -  Training  Features 

Count 

Range 

Mean 

Standard 

Deviation 

Gunnery  uses  repetition  to  train. 

46 

3 

4.32 

0.91 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  uses  repetition  to 
train. 

4.09 

0.72 

c 

QJ 

A  Field  exercise  uses  repetition  to  train 

46 

4 

3.98 

1.11 

E 

QJ 

Ranger  School/Mungadai  training  uses  repetition  to  train. 

46 

3 

3.93 

1.07 

c n 

NTC  uses  repetition  to  train. 

46 

4 

3.89 

1.15 

Basic  training  problem  solving  exercises  uses  repetition  to  train. 

3 

3.74 

0.89 

On-the-job  training  uses  repetition  to  train. 

46 

3 

3.69 

0.97 

Resiliency  training  uses  repetition  to  train. 

46 

4 

3.36 

0.90 

Classroom  training  uses  repetition  to  train. 

46 

4 

3.36 

1.01 

PowerPoint  training  uses  repetition  to  train. 

46 

4 

2.86 

1.32 
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Training  Event 

Quantitative  Results  -  Training  Features 

Count 

Range 

Mean 

Standard 

Deviation 

NTC  incorporates  uncertainty  into  training. 

46 

4 

4.62 

0.89 

Ranger  School/Mungadai  training  incorporates  uncertainty  into  training. 

46 

2 

4.50 

0.65 

A  Field  exercise  incorporates  uncertainty  into  training. 

46 

4 

4.09 

0.91 

e 

QJ 

£ 

QJ 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  incorporates 
uncertainty  into  training. 

3.60 

1.06 

CS 

VI 

Basic  training  problem  solving  exercises  incorporates  uncertainty  into  training. 

4 

3.45 

1.00 

On-the-job  training  incorporates  uncertainty  into  training. 

46 

4 

3.44 

1.08 

Resiliency  training  incorporates  uncertainty  into  training. 

46 

4 

2.85 

0.94 

Gunnery  incorporates  uncertainty  into  training. 

46 

4 

2.80 

1.20 

Classroom  training  incorporates  uncertainty  into  training. 

46 

3 

2.32 

0.88 

PowerPoint  training  incorporates  uncertainty  into  training. 

46 

3 

2.07 

0.91 

D-3 


Training  Event 

Quantitative  Results  -  Training  Features 

Count 

Range 

Mean 

Standard 

Deviation 

Ranger  School/Mungadai  training  incorporates  dealing  with  stress. 

46 

1 

4.67 

0.49 

NTC  incorporates  dealing  with  stress. 

46 

4 

4.65 

0.74 

A  Field  exercise  incorporates  dealing  with  stress. 

46 

3 

4.39 

0.69 

c 

QJ 

E 

QJ 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  incorporates 
dealing  with  stress. 

46 

3 

4.10 

0.85 

C/) 

Gunnery  incorporates  dealing  with  stress. 

46 

3 

3.78 

1.00 

On-the-job  training  incorporates  dealing  with  stress. 

46 

3 

3.73 

0.84 

Basic  training  problem  solving  exercises  incorporates  dealing  with  stress. 

46 

4 

3.59 

0.96 

Resiliency  training  incorporates  dealing  with  stress. 

46 

4 

3.12 

1.12 

Classroom  training  incorporates  dealing  with  stress. 

46 

3 

2.26 

0.91 

PowerPoint  training  incorporates  dealing  with  stress. 

46 

3 

1.98 

0.90 
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Statement 


Training  Event 

_ Quantitative  Results  -  Training  Features _ 

Ranger  School/Mungadai  training  is  a  difficult,  rigorous,  and  challenging 
training  opportunity. 

NTC  is  a  difficult,  rigorous,  and  challenging  training  opportunity. 

A  Field  exercise  is  a  difficult,  rigorous,  and  challenging  training  opportunity. 
Gunnery  is  a  difficult,  rigorous,  and  challenging  training  opportunity. 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  is  a  difficult, 
rigorous,  and  challenging  training  opportunity. 

On-the-job  training  is  a  difficult,  rigorous,  and  challenging  training  opportunity. 

Basic  training  problem  solving  exercises  is  a  difficult,  rigorous,  and  challenging 
training  opportunity. 

Resiliency  training  is  a  difficult,  rigorous,  and  challenging  training  opportunity. 

Classroom  training  is  a  difficult,  rigorous,  and  challenging  training  opportunity. 

PowerPoint  training  is  a  difficult,  rigorous,  and  challenging  training 
opportunity. 
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Training  Event 

Quantitative  Results  -  Training  Features 

Count 

Range 

Mean 

Standard 

Deviation 

Gunnery  focuses  too  much  on  qualifying. 

46 

4 

3.54 

1.27 

Basic  training  problem  solving  exercises  focuses  too  much  on  qualifying. 

3.16 

1.05 

c 

QJ 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  focuses  too  much 
on  qualifying. 

2.85 

1.09 

S 

QJ 

Ranger  School/Mungadai  training  focuses  too  much  on  qualifying. 

4 

2.79 

1.19 

v* 

cn 

Classroom  training  focuses  too  much  on  qualifying. 

46 

4 

2.78 

1.19 

PowerPoint  training  focuses  too  much  on  qualifying. 

46 

4 

2.73 

1.26 

NTC  focuses  too  much  on  qualifying. 

46 

4 

2.62 

1.21 

A  Field  exercise  focuses  too  much  on  qualifying. 

46 

4 

2.52 

1.13 

Resiliency  training  focuses  too  much  on  qualifying. 

46 

3 

2.47 

0.80 

On-the-job  training  focuses  too  much  on  qualifying. 

46 

3 

2.13 

0.92 
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Statement 


Training  Event 

Quantitative  Results  -  Training  Features 

Count 

Range 

Mean 

Standard 

Deviation 

PowerPoint  training  is  generally  a  check  the  box  training  activity. 

3.68 

1.29 

Classroom  training  is  generally  a  check  the  box  training  activity. 

4 

3.58 

1.23 

Resiliency  training  is  generally  a  check  the  box  training  activity. 

4 

3.35 

1.07 

4-* 

C 

QJ 

E 

Basic  training  problem  solving  exercises  is  generally  a  check  the  box  training 
activity. 

46 

4 

2.97 

1.15 

QJ 

+* 

CS 

n 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  is  generally  a  check 
the  box  training  activity. 

46 

4 

2.58 

1.28 

Gunnery  is  generally  a  check  the  box  training  activity. 

46 

4 

2.56 

1.21 

NTC  is  generally  a  check  the  box  training  activity. 

46 

4 

2.28 

1.22 

A  Field  exercise  is  generally  a  check  the  box  training  activity. 

3 

2.27 

1.04 

On-the-job  training  is  generally  a  check  the  box  training  activity. 

4 

2.07 

0.97 

Ranger  School/Mungadai  training  is  generally  a  check  the  box  training  activity. 

46 

2 

1.57 

0.65 
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Statement 


Training  Event 

_ Quantitative  Results  -  Rater  And  Time  Concerns _ 

Ranger  School/Mungadai  training  provides  an  environment  where  peers  can 
accurately  evaluate  perfonnance  on  intangibles. 

NTC  provides  an  environment  where  peers  can  accurately  evaluate  performance 
on  intangibles. 

A  Field  exercise  provides  an  environment  where  peers  can  accurately  evaluate 
performance  on  intangibles. 

On-the-job  training  provides  an  environment  where  peers  can  accurately 
evaluate  performance  on  intangibles. 

Gunnery  provides  an  environment  where  peers  can  accurately  evaluate 
performance  on  intangibles. 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  provides  an 
environment  where  peers  can  accurately  evaluate  perfonnance  on  intangibles. 

Basic  training  problem  solving  exercises  provides  an  environment  where  peers 
can  accurately  evaluate  performance  on  intangibles. 

Resiliency  training  provides  an  environment  where  peers  can  accurately 
evaluate  performance  on  intangibles. 

Classroom  training  provides  an  environment  where  peers  can  accurately 
evaluate  performance  on  intangibles. 

PowerPoint  training  provides  an  environment  where  peers  can  accurately 
evaluate  performance  on  intangibles. 


Count 

Range 

Mean 

Standard 

Deviation 

46 

2 

4.44 

0.81 

3 

4.39 

0.86 

3 

4.16 

0.83 

3 

4.00 

0.86 

4 

3.65 

1.12 

4 

3.49 

0.96 

4 

3.15 

0.94 

4 

2.91 

0.80 

3 

2.66 

0.94 

3 

2.02 

0.99 
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Statement 


Training  Event 

Quantitative  Results  -  Rater  And  Time  Concerns 

Count 

Range 

Mean 

Standard 

Deviation 

Ranger  School/Mungadai  training  provides  a  sufficient  amount  of  time  for 
peers  to  observe  and  critique  performance  on  intangibles. 

46 

2 

4.44 

0.81 

NTC  provides  a  sufficient  amount  of  time  for  peers  to  observe  and  critique 
perfonnance  on  intangibles. 

46 

3 

4.00 

1.03 

On-the-job  training  provides  a  sufficient  amount  of  time  for  peers  to  observe 
and  critique  perfonnance  on  intangibles. 

3 

3.93 

0.93 

A  Field  exercise  provides  a  sufficient  amount  of  time  for  peers  to  observe 
and  critique  perfonnance  on  intangibles. 

46 

3 

3.84 

0.94 

Gunnery  provides  a  sufficient  amount  of  time  for  peers  to  observe  and 
critique  performance  on  intangibles. 

46 

4 

3.63 

0.98 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  provides  a 
sufficient  amount  of  time  for  peers  to  observe  and  critique  performance  on 
intangibles. 

46 

4 

3.42 

1.01 

Basic  training  problem  solving  exercises  provides  a  sufficient  amount  of 
time  for  peers  to  observe  and  critique  performance  on  intangibles. 

46 

4 

3.27 

0.84 

Resiliency  training  provides  a  sufficient  amount  of  time  for  peers  to  observe 
and  critique  perfonnance  on  intangibles. 

3 

2.88 

0.78 

Classroom  training  provides  a  sufficient  amount  of  time  for  peers  to  observe 
and  critique  perfonnance  on  intangibles. 

46 

4 

2.80 

1.07 

PowerPoint  training  provides  a  sufficient  amount  of  time  for  peers  to 
observe  and  critique  performance  on  intangibles. 

46 

4 

2.21 

1.21 
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Statement 


Training  Event 

Quantitative  Results  -  Rater  And  Time  Concerns 

Count 

Range 

Mean 

Standard 

Deviation 

NTC  provides  an  environment  where  superiors  can  accurately  evaluate 
perfonnance  on  intangibles. 

4.39 

0.86 

A  Field  exercise  provides  an  environment  where  superiors  can  accurately 
evaluate  perfonnance  on  intangibles. 

4.18 

0.86 

On-the-job  training  provides  an  environment  where  superiors  can  accurately 
evaluate  perfonnance  on  intangibles. 

4.04 

0.97 

Ranger  School/Mungadai  training  provides  an  environment  where  superiors  can 
accurately  evaluate  performance  on  intangibles. 

4.00 

1.21 

c 

Q J 

S 

QJ 

Gunnery  provides  an  environment  where  superiors  can  accurately  evaluate 
perfonnance  on  intangibles. 

3.98 

1.05 

VI 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  provides  an 
environment  where  superiors  can  accurately  evaluate  perfonnance  on 
intangibles. 

46 

4 

3.60 

0.98 

Basic  training  problem  solving  exercises  provides  an  environment  where 
superiors  can  accurately  evaluate  performance  on  intangibles. 

3 

3.58 

0.94 

Resiliency  training  provides  an  environment  where  superiors  can  accurately 
evaluate  perfonnance  on  intangibles. 

2.97 

0.90 

Classroom  training  provides  an  environment  where  superiors  can  accurately 
evaluate  perfonnance  on  intangibles. 

2.89 

1.17 

PowerPoint  training  provides  an  environment  where  superiors  can  accurately 
evaluate  perfonnance  on  intangibles. 

2.26 

1.29 

Statement 


Training  Event 

_ Quantitative  Results  -  Rater  And  Time  Concerns _ 

NTC  provides  a  sufficient  amount  of  time  for  superiors  to  observe  and  critique 
performance  on  intangibles. 

On-the-job  training  provides  a  sufficient  amount  of  time  for  superiors  to 
observe  and  critique  perfonnance  on  intangibles. 

A  Field  exercise  provides  a  sufficient  amount  of  time  for  superiors  to  observe 
and  critique  performance  on  intangibles. 

Gunnery  provides  a  sufficient  amount  of  time  for  superiors  to  observe  and 
critique  perfonnance  on  intangibles. 

Ranger  School/Mungadai  training  provides  a  sufficient  amount  of  time  for 
superiors  to  observe  and  critique  perfonnance  on  intangibles. 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  provides  a 
sufficient  amount  of  time  for  superiors  to  observe  and  critique  perfonnance  on 
intangibles. 

Basic  training  problem  solving  exercises  provides  a  sufficient  amount  of  time 
for  superiors  to  observe  and  critique  performance  on  intangibles. 

Resiliency  training  provides  a  sufficient  amount  of  time  for  superiors  to 
observe  and  critique  perfonnance  on  intangibles. 

Classroom  training  provides  a  sufficient  amount  of  time  for  superiors  to 
observe  and  critique  perfonnance  on  intangibles. 

PowerPoint  training  provides  a  sufficient  amount  of  time  for  superiors  to 
observe  and  critique  perfonnance  on  intangibles. 


Count 

Range 

Mean 

Standard 

Deviation 

46 

4 

4.09 

1.03 

3.96 

0.99 

46 

4 

3.93 

1.01 

3.78 

1.03 

46 

4 

3.63 

1.31 

46 

4 

3.44 

1.03 

3.21 

0.82 

3.13 

0.91 

2.87 

1.14 

46 

4 

2.26 

1.27 
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Statement 


Training  Event 

Quantitative  Results  -  Rater  And  Time  Concerns 

Count 

Range 

Mean 

Standard 

Deviation 

Ranger  School/Mungadai  training  provides  an  environment  where  Soldiers  can 
accurately  evaluate  their  own  performance  on  intangibles. 

46 

2 

4.19 

0.75 

NTC  provides  an  environment  where  Soldiers  can  accurately  evaluate  their 
own  performance  on  intangibles. 

46 

3 

4.13 

0.98 

A  Field  exercise  provides  an  environment  where  Soldiers  can  accurately 
evaluate  their  own  performance  on  intangibles. 

On-the-job  training  provides  an  environment  where  Soldiers  can  accurately 
evaluate  their  own  performance  on  intangibles. 

Gunnery  provides  an  environment  where  Soldiers  can  accurately  evaluate  their 
own  performance  on  intangibles. 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  provides  an 
environment  where  Soldiers  can  accurately  evaluate  their  own  perfonnance  on 
intangibles. 

Resiliency  Training  provides  an  environment  where  Soldiers  can  accurately 
evaluate  their  own  performance  on  intangibles. 

Basic  training  problem  solving  exercises  provides  an  environment  where 
Soldiers  can  accurately  evaluate  their  own  performance  on  intangibles. 

Classroom  training  provides  an  environment  where  Soldiers  can  accurately 
evaluate  their  own  performance  on  intangibles. 

PowerPoint  training  provides  an  environment  where  Soldiers  can  accurately 
evaluate  their  own  performance  on  intangibles. 


3 

3.96 

0.98 

3 

3.91 

0.90 

46 

4 

3.88 

1.02 

46 

4 

3.65 

1.07 

46 

4 

3.42 

0.87 

46 

4 

3.39 

1.09 

46 

4 

2.84 

1.19 

46 

4 

2.23 

1.21 

E-6 


Statement 


Training  Event 

Quantitative  Results  -  Rater  And  Time  Concerns 


NTC  provides  a  sufficient  amount  of  time  for  Soldiers  to  observe  and  critique 
their  own  performance  on  intangibles. 


Count  Range  Mean 


Standard 

Deviation 


A  Field  exercise  provides  a  sufficient  amount  of  time  for  Soldiers  to  observe 
and  critique  their  own  performance  on  intangibles. 

6 

3 

3.93 

0.89 

On-the-job  training  provides  a  sufficient  amount  of  time  for  Soldiers  to  observe 
and  critique  their  own  performance  on  intangibles. 

■ 

6 

3 

3.91 

0.94 

Gunnery  provides  a  sufficient  amount  of  time  for  Soldiers  to  observe  and 
critique  their  own  performance  on  intangibles. 

1 

6 

3 

3.83 

0.90 

Ranger  School/Mungadai  training  provides  a  sufficient  amount  of  time  for 
Soldiers  to  observe  and  critique  their  own  performance  on  intangibles. 

6 

3 

3.75 

0.93 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  provides  a 
sufficient  amount  of  time  for  Soldiers  to  observe  and  critique  their  own 
performance  on  intangibles. 


Resiliency  training  provides  a  sufficient  amount  of  time  for  Soldiers  to  observe 
and  critique  their  own  performance  on  intangibles. 


Basic  training  problem  solving  exercises  provides  a  sufficient  amount  of  time 
for  Soldiers  to  observe  and  critique  their  own  performance  on  intangibles. 


Classroom  training  provides  a  sufficient  amount  of  time  for  Soldiers  to  observe 
and  critique  their  own  performance  on  intangibles. 


PowerPoint  training  provides  a  sufficient  amount  of  time  for  Soldiers  to 
observe  and  critique  their  own  performance  on  intangibles. 


3.56 

1.10 

3.34 

0.83 

3.30 

0.88 

2.80 

1.24 

2.26 

1.27 
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Statement 


Training  Event 

_ Quantitative  Results  -  Rater  And  Time  Concerns _ 

On-the-job  training  provides  a  sufficient  amount  of  time  for 
superiors/instructors  to  give  individual  feedback. 

Gunnery  provides  a  sufficient  amount  of  time  for  superiors/instructors  to  give 
individual  feedback. 

NTC  provides  a  sufficient  amount  of  time  for  superiors/instructors  to  give 
individual  feedback. 

Ranger  School/Mungadai  training  provides  a  sufficient  amount  of  time  for 
superiors/  instructors  to  give  individual  feedback. 

A  Field  exercise  provides  a  sufficient  amount  of  time  for  superiors/instructors 
to  give  individual  feedback. 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  provides  a 
sufficient  amount  of  time  for  superiors/instructors  to  give  individual  feedback. 
Basic  training  problem  solving  exercises  provides  a  sufficient  amount  of  time 
for  superiors/instructors  to  give  individual  feedback. 

Resiliency  training  provides  a  sufficient  amount  of  time  for 
superiors/instructors  to  give  individual  feedback. 

Classroom  training  provides  a  sufficient  amount  of  time  for 
superiors/instructors  to  give  individual  feedback. 

PowerPoint  training  provides  a  sufficient  amount  of  time  for 
superiors/instructors  to  give  individual  feedback. 


Count 

Range 

Mean 

Standard 

Deviation 

46 

3 

4.11 

0.83 

46 

3 

4.10 

0.78 

46 

4 

4.02 

1.04 

46 

4 

3.88 

1.15 

3 

3.80 

0.98 

46 

4 

3.67 

0.94 

46 

4 

3.45 

0.97 

46 

4 

3.22 

0.79 

46 

4 

3.11 

1.32 

46 

4 

2.70 

1.42 
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Statement 


Training  Event 

Quantitative  Results  -  Intangibles 

Count 

Range 

Mean 

Standard 

Deviation 

NTC  is  an  excellent  environment  to  measure  Soldier's  initiative. 

4 

4.63 

0.71 

Ranger  School/Mungadai  training  is  an  excellent  environment  to  measure 

Soldier's  initiative. 

4.63 

0.62 

A  Field  exercise  is  an  excellent  environment  to  measure  Soldier's  initiative. 

46 

2 

4.56 

0.59 

a 

QJ 

£ 

QJ 

On-the-job  training  is  an  excellent  environment  to  measure  Soldier's  initiative. 

46 

4 

4.31 

0.82 

Gunnery  is  an  excellent  environment  to  measure  Soldier's  initiative. 

46 

3 

3.88 

0.97 

-*-* 

Ifl 

Basic  training  problem  solving  exercises  is  an  excellent  environment  to  measure 
Soldier's  initiative. 

3 

3.76 

0.75 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  is  an  excellent 
environment  to  measure  Soldier’s  initiative. 

3.65 

0.97 

Resiliency  training  is  an  excellent  environment  to  measure  Soldier's  initiative. 

46 

4 

2.97 

1.00 

Classroom  training  is  an  excellent  environment  to  measure  Soldier's  initiative. 

46 

4 

2.52 

1.07 

PowerPoint  training  is  an  excellent  environment  to  measure  Soldier's  initiative. 

46 

4 

2.19 

1.10 
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Statement 


Training  Event 

Quantitative  Results  -  Intangibles 

Count 

Range 

Mean 

Standard 

Deviation 

Ranger  School/Mungadai  training  is  an  excellent  environment  to  measure 

Soldier's  will. 

46 

1 

4.69 

0.48 

NTC  is  an  excellent  environment  to  measure  Soldier's  will. 

4 

4.65 

0.71 

A  Field  exercise  is  an  excellent  environment  to  measure  Soldier's  will. 

46 

1 

4.61 

0.49 

c 

QJ 

On-the-job  training  is  an  excellent  environment  to  measure  Soldier's  will. 

46 

4 

4.13 

0.97 

B 

QJ 

Gunnery  is  an  excellent  environment  to  measure  Soldier's  will. 

3 

3.90 

0.90 

Ifl 

Basic  training  problem  solving  exercises  is  an  excellent  environment  to  measure 
Soldier's  will. 

46 

4 

3.61 

0.93 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  is  an  excellent 
environment  to  measure  Soldier’s  will. 

46 

4 

3.53 

1.08 

Resiliency  training  is  an  excellent  environment  to  measure  Soldier's  will. 

4 

3.09 

0.93 

Classroom  training  is  an  excellent  environment  to  measure  Soldier's  will. 

46 

3 

2.27 

1.00 

PowerPoint  training  is  an  excellent  environment  to  measure  Soldier's  will. 

46 

3 

2.00 

0.93 

F-3 


Statement 


Training  Event 

Quantitative  Results  -  Intangibles 

Count 

Range 

Mean 

Standard 

Deviation 

Ranger  School/Mungadai  training  is  an  excellent  environment  to  measure 
Soldier's  grit. 

46 

1 

4.63 

0.50 

NTC  is  an  excellent  environment  to  measure  Soldier's  grit. 

4 

4.59 

0.80 

A  Field  exercise  is  an  excellent  environment  to  measure  Soldier's  grit. 

46 

3 

4.42 

0.72 

c 

QJ 

On-the-job  training  is  an  excellent  environment  to  measure  Soldier's  grit. 

46 

4 

4.11 

1.03 

£ 
a j 
-w 

Gunnery  is  an  excellent  environment  to  measure  Soldier's  grit. 

46 

3 

3.73 

1.06 

m 

Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  is  an  excellent 
environment  to  measure  Soldier’s  grit. 

46 

4 

3.47 

1.01 

Basic  training  problem  solving  exercises  is  an  excellent  environment  to 
measure  Soldier's  grit. 

46 

4 

3.45 

1.00 

Resiliency  training  is  an  excellent  environment  to  measure  Soldier's  grit. 

46 

4 

3.03 

1.00 

Classroom  training  is  an  excellent  environment  to  measure  Soldier's  grit. 

46 

3 

2.20 

0.98 

PowerPoint  training  is  an  excellent  environment  to  measure  Soldier's  grit. 

46 

3 

1.98 

0.91 

Statement 


Training  Event 

_ Quantitative  Results  -  Intangibles _ 

Ranger  School/Mungadai  training  is  an  excellent  environment  to  measure 

Soldier’s  hardiness. _ 

NTC  is  an  excellent  environment  to  measure  Soldier's  hardiness. 


Count  Range 

46  1 


A  Field  exercise  is  an  excellent  environment  to  measure  Soldier's  hardiness. 
On-the-job  training  is  an  excellent  environment  to  measure  Soldier's  hardiness. 


Gunnery  is  an  excellent  environment  to  measure  Soldier's  hardiness. 


Basic  training  problem  solving  exercises  is  an  excellent  environment  to  measure 
Soldier's  hardiness. 


Mean 

4.75 

4.67 

4.52 
4.07 
3.73 

3.52 


Combat  Life-Saver  training/  MSTC  training/  Trauma  Lane  is  an  excellent 
environment  to  measure  Soldier's  hardiness. 

Resiliency  training  is  an  excellent  environment  to  measure  Soldier’s  hardiness. 
Classroom  training  is  an  excellent  environment  to  measure  Soldier's  hardiness. 
PowerPoint  training  is  an  excellent  environment  to  measure  Soldier's  hardiness 


3.47 

3.00 

2.25 

2.02 


Standard 

Deviation 

0.45 

0.73 

0.63 

0.94 

1.06 

0.97 

1.08 

0.95 

0.94 

0.91 
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Theme  # 

Theme 

Research  Question:  1  Training  Methods  and  Learning  Theories  Used  in  Existing 
Exercises,  Venues,  and  Activities 

1.1 

Training  Methods 

1.1.1 

Practice/Experience 

1.1.2 

Realism 

wm 

Multiple,  different  opportunities  to  apply  KSAs 

ESEBS 

Collaborative  or  Team-based 

1.1.5 

Explaining  grasp  of  training  exercise 

1.1.6 

Providing  a  support  structure  (e.g.,  OC's,  Instructors,  Evaluators) 

1.1.7 

Involves  problem  solving  and  making  sense  of  environment 

1.1.8 

Self-Examination  or  Self-Assessment 

1.1.9 

Crawl-Walk-Run;  or  Establishing  a  Baseline  and  then  Introducing 

Complexity 

1.1.10 

AARs/Group  or  Team  Feedback 

1.1.11 

Deployment/Combat  Experience 

1.1.12 

Role  Modeling/Leading  by  Example 

1.1.13 

Individual  Developmental  Feedback/Counseling/Mentorship 

1.1.14 

Other  Training  Methods 

1.2 

Best  Training  Features 

1.2.1 

Repetition 

1.2.2 

Incorporating  Uncertainty 

1.2.3 

Dealing  with  Stress/Pressure 

1.2.4 

Difficult,  Rigorous,  Challenging  Training 

1.2.5 

Incorporating  Problem  Solving 

1.2.6 

Setbacks  or  Unforeseen  Hurdles 

1.2.7 

Other  Training  Features 

1.2.8 

Adverse  Weather  or  Conditions 

1.3 

Worst  Training  Features 

1.3.1 

Focuses  too  much  on  qualifying 

1.3.2 

Check  the  box  training 
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Theme  # 

Theme 

Research  Question  2:  Measurement  Characteristics  that  are  Most  Appropriate  and 

How  Measurement  of  Intangibles  Can  be  Accomplished  and  Adapted  to  the  Field 

2.1 

Best  Training  Environment 

2.1.1 

FTX/STX  Lanes 

2.1. 1.1 

Unit  Level  Training 

2.1. 1.2 

Platoon  Level  Training 

2.1. 1.3 

Squad  Level  Training 

2.1. 1.4 

Team  Level  Training 

2.1.2 

NTC/JRTC/CTC 

2.1.3 

Gunnery 

2.1.4 

Resiliency  Training 

2.1.5 

CLS/MSTC 

2.1.6 

Ranger/Mungadai 

2.1.7 

Basic  Combat  Training 

2.1.8 

Classroom  Training 

2.1.9 

PPT  Slides 

2.1.10 

OJT/On  the  Job 

2.1.11 

PT  /  Fitness 

2.1.12 

Combatives  Training 

2.1.13 

Other  Training  Environment 

2.2 

How  is  Training  Currently  or  Best  Evaluated? 

2.2.1 

AAR 

2.2.2 

Instructor/Observer  Controller 

2.2.3 

Self-Evaluation 

2.2.4 

Subordinate  Evaluation 

2.2.5 

Peer  Evaluation 

2.2.6 

Superior  Evaluation 

2.2.6. 1 

First  Line  Leader  (FLL) 

22.6.2 

Unit  Level/Commander 

22.6.3 

Platoon  Leader/Level 

22.6.4 

Squad  Leader/Level 

22.6.5 

Team  Leader/Level 
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Theme  # 

Theme 

Research 

Question  3:  Training  Performance  Indicators  for  the  Selected  Intangibles 

3.1 

Initiative  Indicators 

3.1.1 

Knows  responsibilities/takes  or  assumes  responsibility/ownership 

3.1.2 

Listens/Understands  Big  Picture/Commanders  intent 

3.1.3 

Anticipates  what  needs  to  be  done 

3.1.4 

Prepares  for  Next  Step 

3.1.5 

Effectively  Uses  Time 

3.1.6 

Leads  Peers 

3.1.7 

Other  -  Initiative  Indictor 

3.2 

Will  Indicators 

3.2.1 

Persistence/How  a  Soldier  reacts  to  adversity 

3.2.2 

Lack  of  Complaint/  Can-Do  Attitude 

3.2.3 

Demonstrating  a  Will  to  Win 

3.2.4 

Other  -  Will  Indicator 

Grit  Indicators 

Persistence 

Interested  in  learning  other  jobs 

Dedication  to  the  Army 

Other  -  Grit  Indicator 

3.4 

Hardiness  Indicators 

3.4.1 

Persistence 

3.4.2 

Other  -  Hardiness  Indicator 

Other  Cross  Question  Themes 

4.1 

The  source  of  intangibles  is  values  or  trait-based 

4.2 

Not  enough  time/resources  for  training 

4.3 

One  or  more  of  the  intangibles  seem  related 

4.4 

Intangibles  can  be  tested  at  the  same  time 
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